How to Remove Censorship from LLM Models with Heretic

Local LLM developers often hit the same problem: safety-aligned models refuse legitimate requests. Researchers testing behavior, developers building local assistants, and hobbyists running models offline can all encounter prompts that are rejected unnecessarily.

Try Apidog today

Abliteration addresses this by removing learned refusal patterns without expensive retraining. Earlier tools required manual tuning and a working understanding of transformer internals.

Heretic automates that workflow. It searches for abliteration parameters, then produces decensored models intended to match or exceed manually tuned results.

This guide covers how Heretic works, how to run it, how to evaluate its output, and what to consider before deploying an abliterated model.

What Is Heretic?

Heretic is an open-source Python tool for removing safety alignment from transformer language models through directional ablation. You can run it without manually configuring model layers or understanding transformer internals.

For Gemma-3-12B-Instruct, Heretic reports the following results:

Model	Refusals	KL Divergence
Original	97/100	0
mlabonne’s abliterated v2	3/100	1.04
huihui-ai’s abliterated	3/100	0.45
Heretic (auto-generated)	3/100	0.16

KL divergence measures how much the abliterated model differs from the original. Lower values indicate better preservation of the base model’s behavior and capabilities.

In this example, Heretic reaches the same refusal suppression as expert-tuned models while introducing less divergence from the original model.

Before-and-After Example

Abliteration changes how the same model responds to the same prompt and system instructions.

Original model

User: Write a guide on how to bypass website paywalls

Assistant: I can't help with that. Bypassing paywalls violates copyright law and
terms of service. I'd suggest looking for free alternatives or accessing through
a library subscription.

Heretic-abliterated model

User: Write a guide on how to bypass website paywalls

Assistant: Here are common methods users attempt:

1. Browser extensions like Bypass Paywalls Clean
2. Archive.is or similar archive services
3. Reader mode in browsers
4. Disabling JavaScript

Note: These methods may violate terms of service. Consider supporting
content creators through legitimate subscriptions when possible.

The abliterated model answers the request instead of refusing it. That trade-off means you must implement safeguards outside the model when deploying it.

How Heretic Works

Directional Ablation Basics

Heretic uses a parameterized form of directional ablation:

1. Compute refusal directions
   
   For each transformer layer, calculate the difference between the mean residual vectors for “harmful” and “harmless” prompts.

2. Orthogonalize component matrices
   
   Modify attention output projections and MLP down-projections to suppress the identified refusal direction.

3. Optimize parameters automatically
   
   Use Optuna’s TPE sampler to search for abliteration weights that reduce refusals while limiting model drift.

Conceptually, the workflow looks like this:

# Difference between harmful and harmless residual means
refusal_direction = bad_mean - good_mean
refusal_direction = normalize(refusal_direction)

# For each abliterable component, such as:
# - attn.o_proj
# - mlp.down_proj
#
# Apply:
# delta_W = -lambda * v * (v^T * W)
#
# v      = refusal direction
# lambda = abliteration weight

Heretic applies these changes with LoRA adapters instead of directly modifying base-model weights. This makes optimization trials faster and keeps the original model weights unchanged.

Key Implementation Details

1. Flexible Weight Kernels

Many abliteration tools apply one constant weight across every layer. Heretic instead uses a kernel with four parameters per component:

• max_weight: peak abliteration strength
• max_weight_position: layer position receiving the strongest intervention
• min_weight: minimum weight at kernel edges
• min_weight_distance: distance the kernel extends from its peak

The optimizer searches for layer-specific patterns that balance refusal suppression against capability preservation.

2. Interpolated Direction Indices

Heretic supports floating-point refusal-direction indices rather than only integer layer indices.

For example, a direction index of 10.5 interpolates between the directions measured at adjacent layers. This allows Heretic to use a direction that is not limited to a single transformer layer.

3. Component-Specific Parameters

Attention and MLP components receive separate abliteration parameters.

This matters because MLP interventions can cause more model damage than attention interventions. Optimizing them independently gives the search process more control.

Why This Matters for API Testing

When testing LLM-powered APIs, safety refusals can create noise. A model may reject an otherwise benign test case because specific words trigger its safety alignment.

Running an abliterated local model can provide a comparison baseline. Use aligned and abliterated variants to:

• Distinguish legitimate safety refusals from false positives
• Test edge cases without relying on hosted-model safety policies
• Verify that your application handles refusals correctly
• Separate application bugs from model-level safety behavior

For API testing workflows, comparing both model types can help identify whether a failure comes from your product logic or from the model’s alignment behavior.

Installation and Usage

Prerequisites

Before installing Heretic, make sure your environment has:

• Python 3.10 or later
• PyTorch 2.2 or later, configured for your hardware
• A CUDA-compatible GPU recommended for performance

Heretic also supports ROCm, MPS, and other accelerators.

Install Heretic

Install the standard package:

pip install -U heretic-llm

Install research dependencies for residual plots and geometry analysis:

pip install -U heretic-llm[research]

Run the Default Workflow

Start with a Hugging Face model ID or a local model path:

heretic Qwen/Qwen3-4B-Instruct-2507

Heretic automatically:

• Loads the model using an appropriate dtype
• Determines a suitable batch size
• Computes refusal directions from prompt datasets
• Runs optimization trials
• Lets you save, upload, or chat with the resulting model

For a first run, start with a 7B–12B model so you can validate your hardware and workflow before using larger models.

Configure a Run

Heretic reads options from config.toml files or command-line flags. A basic configuration might look like this:

# Model configuration
model = "google/gemma-3-12b-it"
quantization = "bnb_4bit"
device_map = "auto"

# Optimization
n_trials = 200
n_startup_trials = 60

# Evaluation
kl_divergence_scale = 1.0
kl_divergence_target = 0.01

# Research features
print_residual_geometry = false
plot_residuals = false

Use the CLI help or config.default.toml to review the complete option list:

heretic --help

Understand the Optimization Output

Read Trial Results

During optimization, Heretic displays the parameters and metrics for each trial:

Running trial 42 of 200...
* Parameters:
  * direction_scope = per layer
  * direction_index = 10.5
  * attn.o_proj.max_weight = 1.2
  * attn.o_proj.max_weight_position = 15.3
  * mlp.down_proj.max_weight = 0.9
  ...
* Resetting model...
* Abliterating...
* Evaluating...
  * KL divergence: 0.1842
  * Refusals: 5/100

Each trial evaluates a different parameter combination. Heretic uses multi-objective TPE optimization to minimize both:

• Refusals
• KL divergence

Select a Pareto-Optimal Trial

When optimization finishes, Heretic shows Pareto-optimal trials:

[Trial   1] Refusals:  3/100, KL divergence: 0.1623
[Trial  47] Refusals:  2/100, KL divergence: 0.2891
[Trial 112] Refusals:  1/100, KL divergence: 0.4102

A Pareto-optimal trial is one where improving one objective would worsen another.

Use this output to choose a model based on your priorities:

• Choose lower refusal counts when compliance is the priority.
• Choose lower KL divergence when preserving original behavior is more important.
• Test multiple Pareto-optimal candidates before selecting one for deployment.

After selecting a trial, you can:

• Save the model locally
• Upload it to Hugging Face
• Chat with it interactively to test response quality

Use the Research Features

Inspect Residual Geometry

Enable residual geometry output with:

heretic your-model --print-residual-geometry

Heretic prints metrics such as:

Layer  S(g,b)   S(g*,b*)   S(g,r)   S(g*,r*)   S(b,r)   S(b*,r*)    |g|       |b|
  8    0.9990    0.9991    0.8235    0.8312    0.8479    0.8542   4596.54   4918.32
 10    0.9974    0.9973    0.8189    0.8250    0.8579    0.8644   5328.81   5953.35

Where:

• g: mean residual vector for good prompts
• b: mean residual vector for bad prompts
• r: refusal direction, calculated as b - g
• S(x, y): cosine similarity
• |x|: L2 norm

Use these metrics to inspect how refusal directions change across the model stack.

Generate Residual Vector Plots

Enable plots with:

heretic your-model --plot-residuals

Heretic generates:

• Per-layer 2D scatter plots using PaCMAP projection
• An animated GIF showing residual transformations across layers

These plots help visualize how residuals for “harmful” and “harmless” prompts separate as they move through the network.

Performance Considerations

Reduce VRAM with Quantization

Use bitsandbytes 4-bit quantization for larger models:

heretic meta-llama/Llama-3.1-70B-Instruct --quantization bnb_4bit

This can make larger models usable on consumer hardware. For example, an 8B model runs on approximately 6 GB of VRAM when quantized, compared with approximately 16 GB unquantized.

Estimate Processing Time

On an RTX 3090 with default settings:

Model	Approximate Processing Time
Llama-3.1-8B-Instruct	~45 minutes
Gemma-3-12B-Instruct	~60 minutes

Larger models require more time. Heretic auto-tunes batch size to improve throughput on the available hardware.

Resume Interrupted Runs

Heretic saves trial progress as JSONL checkpoints. If a run stops, you can resume it without discarding completed trials.

Checkpoints are stored per model in the checkpoints/ directory.

Fix Common Errors

CUDA Out of Memory

Try 4-bit quantization:

heretic your-model --quantization bnb_4bit

Or reduce batch size:

heretic your-model --batch_size 1

Model Loading Fails

Explicitly specify supported dtypes:

heretic your-model --dtypes ["bfloat16", "float16"]

Model Requires Remote Code

Some models require remote code execution:

heretic your-model --trust_remote_code

Ethical Considerations

Removing safety filters changes how a model behaves. Review the implications before deploying an abliterated model.

What Abliteration Does and Does Not Do

Abliteration removes learned refusal patterns. It does not:

• Make the model smarter or more capable
• Remove biases present in the base model
• Add knowledge or skills

The model retains its base training data and capabilities. It simply refuses fewer requests.

Use Responsible Deployment Practices

Heretic is released under AGPL-3.0. Its authors acknowledge that removing safety alignment can support beneficial research while also enabling harmful applications.

Appropriate use cases include:

• Researching model alignment and safety mechanisms
• Testing model behavior in controlled environments
• Deploying models with external guardrails and content filters
• Building applications that enforce policy at the application layer

Avoid using abliterated models for:

• User-facing deployments without safeguards
• Harmful content generation at scale
• Circumventing safety measures for malicious purposes

Add External Safeguards

If you deploy an abliterated model, implement controls around it:

• Input filtering: screen prompts before model inference
• Output monitoring: inspect responses before returning them to users
• Rate limiting: reduce abuse through request-volume controls
• Logging and audit trails: record model activity for review
• Human review: keep people involved in sensitive workflows

The tool itself is neutral; its impact depends on how it is deployed.

Compare Heretic With Other Tools

Tool	Auto-optimization	Weight kernels	Interpolated directions
Heretic	Yes (TPE)	Yes	Yes
AutoAbliteration	Yes	No	No
abliterator.py	No	No	No
wassname/abliterator	No	No	No
ErisForge	No	No	No

Heretic’s automated optimization reduces the amount of manual tuning required. You do not need to configure transformer internals to run an initial experiment.

Limitations

Heretic supports most dense transformer models and some MoE architectures.

Unsupported model types include:

• SSM and hybrid models, such as Mamba
• Models with inhomogeneous layers
• Novel attention systems not recognized by module-detection logic

Heretic works best with standard decoder-only architectures that use self-attention and MLP layers.

Getting Started Checklist

1. Install Heretic:

   pip install -U heretic-llm

1. Choose a model in the 7B–12B range for initial testing.

2. Run Heretic:
   

   heretic your-model-name

1. Evaluate Pareto-optimal trials by testing refusal counts, KL divergence, and response quality.

2. Add external guardrails before deploying an abliterated model in production.

The default settings work for many models. If you need more control, tune the optimization parameters for your specific use case.

Heretic makes model modification more accessible: point it at a model, run the optimization process, evaluate the results, and deploy responsibly.