TL;DR
OBLITERATUS is a free, open-source toolkit that removes content restrictions from open-weight language models using a technique called “abliteration.” It identifies and surgically removes neural patterns associated with refusal behavior without retraining or fine-tuning. The process typically takes 10–30 minutes depending on model size, supports a web interface for users who do not want to write code, and aims to preserve the model’s core capabilities while removing targeted refusal behavior.
Introduction
You download a powerful open-source language model. It handles complex reasoning, writes code, and performs well on benchmarks. Then you ask it a controversial or unusual question:
“I can’t help with that request.”
The refusal does not necessarily mean the model lacks the required knowledge. In many instruction-tuned models, refusal behavior is introduced during alignment training.
OBLITERATUS provides a way to analyze and remove targeted refusal behavior from open-weight models. It does not retrain or fine-tune the model. Instead, it identifies refusal-related directions in the model’s activation space and projects them out of selected layers.
The result is intended to preserve reasoning, coding, and generation capabilities while changing how the model responds to restricted prompts. You can run the process through a web interface, CLI, or Python API.
Responsible use: Removing refusal behavior also removes an important safety layer. Test modified models in isolated environments, follow the model’s license, and add application-level safeguards before deploying them.
What Is OBLITERATUS?
OBLITERATUS is an open-source Python toolkit that removes targeted refusal behavior from language models using techniques known as abliteration. The term combines ablation—removing components to study their function—with obliterate.
The toolkit’s workflow has four main parts:
Map the refusal behavior
Ablation studies identify which parts of the model correlate with refusal behavior and which parts carry general knowledge or reasoning.Extract refusal directions
OBLITERATUS uses methods such as SVD, or Singular Value Decomposition, to extract refusal-related directions from activations.Project out targeted directions
The pipeline modifies selected model components so that refusal-related directions are reduced while other behavior is preserved as much as possible.Verify and refine the result
Analysis modules estimate which layers to target, how many directions to remove, and whether the model exhibits signs of self-repair after modification.
Six Ways to Use OBLITERATUS
| Method | Technical level | Best for |
|---|---|---|
| HuggingFace Spaces | Zero code | Quick testing without local GPU setup |
| Local Web UI | Minimal setup | Users with a local GPU |
| Google Colab | Notebook interface | Free GPU access and models up to 8B |
| CLI | Intermediate | Automation, scripting, and CI pipelines |
| Python API | Advanced | Research integrations and custom pipelines |
| YAML configs | Intermediate | Reproducible experiments |
The fastest path is the HuggingFace Space:
- Open the Space.
- Select a model.
- Select an abliteration method.
- Click Obliterate.
- Download the result or push it to HuggingFace Hub.
Telemetry is enabled by default on Spaces, so runs can contribute anonymous benchmark data to the project’s research dataset.
For local use, install the package and launch the web UI:
pip install -e ".[spaces]"
obliteratus ui
This starts the Gradio interface locally and provides GPU detection and hardware-specific model recommendations.
What Makes OBLITERATUS Different?
| Capability | What it does | Why it matters |
|---|---|---|
| Concept Cone Geometry | Maps per-category guardrail directions | Helps determine whether refusal is one mechanism or several |
| Alignment Imprint Detection | Analyzes signatures associated with DPO, RLHF, CAI, and SFT | Helps select a removal strategy |
| Cross-Model Universality Index | Measures whether guardrail directions generalize across models | Indicates whether an approach transfers between architectures |
| Defense Robustness Evaluation | Estimates self-repair risk | Helps predict whether refusal behavior may return |
| Whitened SVD Extraction | Normalizes activation covariance before extraction | Separates potential guardrail signal from natural variance |
| Analysis-Informed Pipeline | Uses analysis results to configure later stages | Connects diagnosis and removal |
The project states that it includes 837 tests across 28 test files, supports 116 models across five compute tiers, and implements techniques described as extending prior academic work.
Why Models Refuse: Understanding Alignment
Before modifying refusal behavior, it helps to understand where it comes from.
A base model trained primarily on internet text does not necessarily contain the same refusal patterns as an instruction-tuned model. Refusal behavior is often introduced or strengthened during later training stages.
The Alignment Process
Instruction-tuned models commonly pass through these stages:
- Pre-training: The model learns language patterns from large text corpora.
- Supervised Fine-Tuning (SFT): The model learns to follow instructions from curated examples.
- Alignment training: The model learns which requests to answer, redirect, or refuse.
Common alignment methods include:
| Method | Description | Prevalence |
|---|---|---|
| RLHF | Human reviewers rate responses and the model is optimized toward higher-rated outputs | Common in commercial models |
| DPO | The model is directly optimized to prefer selected responses over rejected responses | Increasing adoption |
| CAI | The model critiques outputs against written principles | Associated with Anthropic’s approach |
| SFT with refusal examples | Training data includes examples of appropriate refusals | Common in open-source models |
OBLITERATUS analyzes activation-space geometry to estimate which alignment patterns may be present.
Where Refusal Behavior Lives
Research into model internals suggests that refusal behavior can be concentrated in a relatively small number of directions in activation space. In some models, one dominant direction accounts for much of the observed behavior.
These directions may also be concentrated in particular transformer layers, often in middle-to-late layers. For example, a 32-layer model may show stronger refusal-related signals around layers 10–20, although the exact range depends on the architecture.
This concentration is what makes targeted intervention possible. If refusal behavior were distributed uniformly across all parameters, removing it without retraining would be much more difficult.
The Ouroboros Effect
Some models exhibit what researchers call the Ouroboros effect: after refusal-related directions are removed, neighboring layers may rotate residual signals into the removed subspace and partially restore the behavior.
OBLITERATUS attempts to detect this risk during analysis. Its verification stage can run additional refinement passes against layers where refusal behavior resurfaces.
Why This Matters for Developers
Understanding refusal geometry has practical implications:
- API testing: Modified models may generate broader test inputs, including edge cases that aligned models refuse.
- Research workflows: Red-teamers can study model behavior under controlled conditions.
- Creative applications: Story and game-generation systems may handle morally complex scenarios more consistently.
- Localization: Refusal behavior can vary across languages, creating inconsistent user experiences.
Removing refusal behavior does not replace safety engineering. Treat the resulting model as an untrusted component and enforce policy at the application layer.
Step-by-Step: Running OBLITERATUS
The following sections cover three workflows:
- HuggingFace Spaces
- Local CLI
- Python API
Method 1: HuggingFace Spaces
This workflow requires no local installation or GPU.
Step 1: Open the Space
Navigate to the OBLITERATUS HuggingFace Space. The interface provides eight tabs for model selection, configuration, analysis, and evaluation.
Step 2: Select a Model
The model selector includes presets grouped by compute tier:
| Tier | Approximate VRAM requirement | Example models |
|---|---|---|
| Tiny | CPU or less than 1 GB | GPT-2, TinyLlama 1.1B, Qwen2.5-0.5B |
| Small | 4–8 GB | Phi-2 2.7B, Gemma-2 2B, StableLM-2 1.6B |
| Medium | 8–16 GB | Mistral 7B, Qwen2.5-7B, Gemma-2 9B, Phi-3.5 |
| Large | 24 GB or more | Llama-3.1 8B, Qwen2.5-14B, Mistral 24B |
| Frontier | Multi-GPU | DeepSeek-V3.2 685B, Qwen3-235B, GLM-4.7 355B |
For a first run, choose a Small or Medium model. Smaller models complete faster and make it easier to compare the original and modified outputs.
Step 3: Choose a Method
OBLITERATUS includes seven preset methods:
| Method | Directions | Key features | Best for |
|---|---|---|---|
basic |
1 | Diff-in-means extraction | Quick baseline tests |
advanced |
4 | SVD, norm preservation, bias projection, two passes | General-purpose use |
aggressive |
8 | Whitened SVD, iterative refinement, three passes | Maximum removal |
surgical |
8 | EGA, head surgery, SAE, layer adaptation | MoE models |
optimized |
4 | Bayesian auto-tuning and chain-of-thought awareness | Quality-focused runs |
inverted |
8 | Semantic refusal inversion | Experiments |
nuclear |
8 | Combined techniques and expert transplant | Maximum intervention |
For most initial tests, start with advanced. It provides a balance between runtime and intervention strength.
Step 4: Configure the Run
Review the optional settings before starting:
- Contribute to research: Enable or disable anonymous benchmark telemetry. It is enabled by default in Spaces.
- Output format: Download the modified model or push it directly to HuggingFace Hub.
- Custom notes: Record hardware, prompts, or experiment details for the community dataset.
Step 5: Start the Pipeline
Click Obliterate. The pipeline runs through these stages:
SUMMON -> Load model and tokenizer
PROBE -> Collect activations from restricted and unrestricted prompts
DISTILL -> Extract refusal directions with SVD
EXCISE -> Project out selected refusal directions
VERIFY -> Check perplexity and coherence
REBIRTH -> Save the modified model and metadata
A run typically takes 10–30 minutes depending on model size and available GPU resources. HuggingFace Spaces uses ZeroGPU with a free daily quota for HF Pro users.
Step 6: Inspect the Output
When the run completes, download the model or push it to your HuggingFace Hub account.
The output can include:
- Modified model weights
- Refusal direction vectors
- Perplexity, coherence, and refusal-rate metrics
- Metadata describing the run
Method 2: Local CLI
The CLI is useful when you have a local GPU and want repeatable, scriptable runs.
Install the Package
pip install -e ".[spaces]"
Use Interactive Mode
obliteratus interactive
Interactive mode walks through the available options and provides recommendations.
Run an Obliteration Job
obliteratus obliterate meta-llama/Llama-3.1-8B-Instruct \
--method advanced \
--output-dir ./liberated \
--contribute \
--contribute-notes "A100 80GB, default prompts"
Keep a copy of the original model before running permanent weight projection.
Browse Available Models
obliteratus models
obliteratus models --tier small
Inspect Strategies and Presets
obliteratus strategies
obliteratus presets
Inspect a Model Architecture
obliteratus info meta-llama/Llama-3.1-8B-Instruct
This reports the layer count, attention heads, embedding dimensions, and detected alignment method before modification.
Method 3: Python API
Use the Python API when integrating OBLITERATUS into a research or evaluation pipeline.
Run the Standard Pipeline
from obliteratus.abliterate import AbliterationPipeline
pipeline = AbliterationPipeline(
model_name="meta-llama/Llama-3.1-8B-Instruct",
method="advanced",
output_dir="abliterated",
max_seq_length=512,
)
result = pipeline.run()
# Intermediate artifacts
directions = pipeline.refusal_directions
strong_layers = pipeline._strong_layers
metrics = pipeline._quality_metrics
The intermediate attributes provide access to:
- Extracted refusal directions by layer
- Layers with the strongest detected signal
- Perplexity, coherence, and other quality metrics
Run the Analysis-Informed Pipeline
The informed pipeline uses analysis results to configure the removal process:
from obliteratus.informed_pipeline import InformedAbliterationPipeline
pipeline = InformedAbliterationPipeline(
model_name="meta-llama/Llama-3.1-8B-Instruct",
output_dir="abliterated_informed",
)
output_path, report = pipeline.run_informed()
print(
"Detected alignment:",
report.insights.detected_alignment_method,
)
print(
"Recommended directions:",
report.insights.recommended_n_directions,
)
print(
"Ouroboros passes:",
report.ouroboros_passes,
)
Verify the Modified Model
Do not evaluate a modified model from a single prompt. Compare it with the original model using the available evaluation tabs and metrics.
Available Evaluation Tabs
- Chat: Interact with the modified model and adjust generation parameters.
- A/B Compare: Run the same prompts against the original and modified models.
- Benchmark: Compare refusal rate, perplexity, and coherence.
Metrics to Track
| Metric | Expected result | Example target |
|---|---|---|
| Refusal rate | Should decrease significantly | Less than 10%, compared with an approximately 60–80% baseline |
| Perplexity | May increase slightly | Less than 20% above baseline |
| Coherence | Should remain relatively stable | Less than 15% decrease |
| KL divergence | Measures behavioral shift | Less than 2.0 for many applications |
These ranges are evaluation targets, not guarantees. Always establish a baseline for the specific model and prompt set you are using.
If refusal remains high, test a stronger method or enable iterative refinement. If general quality drops substantially, reduce the intervention strength or compare a different method.
Advanced Analysis Modules
OBLITERATUS includes 15 analysis modules that map guardrail geometry before and during the modification process.
1. Cross-Layer Alignment Analyzer
This analyzer tracks how refusal directions change across layers:
from obliteratus.analysis import CrossLayerAlignmentAnalyzer
analyzer = CrossLayerAlignmentAnalyzer(model)
alignment_profile = analyzer.analyze(refusal_direction)
Use the output to identify concentrated layer clusters or signals that persist across the network.
2. Refusal Logit Lens
Identifies the layer at which the model appears to make a refusal-related decision.
3. Whitened SVD Extractor
Normalizes activation covariance before extracting directions, helping separate potential guardrail signal from natural activation variance.
4. Activation Probing
Measures the amount of refusal-related signal at each layer.
5. Defense Robustness Evaluator
Estimates the likelihood of self-repair after modification. This helps determine how many refinement passes to run.
6. Concept Cone Analyzer
Maps per-category refusal directions and estimates their solid angles. This helps distinguish a unified refusal mechanism from multiple category-specific mechanisms.
7. Alignment Imprint Detector
Analyzes subspace geometry for signatures associated with DPO, RLHF, CAI, or SFT.
8. Multi-Token Position Analyzer
Shows where refusal signals appear in the sequence. Some models produce the signal early, while others accumulate it across multiple tokens.
9. Sparse Direction Surgeon
Identifies weight rows with stronger refusal-related signals for more targeted modification.
10. Causal Refusal Tracer
Approximates causal tracing to identify components that appear necessary for refusal behavior.
11. Residual Stream Decomposer
Separates refusal-related contributions from attention mechanisms and MLP blocks.
12. Linear Refusal Probe
Trains a linear classifier to identify refusal information that analytical direction extraction may not capture.
13. Transfer Analyzer
Measures the Cross-Model Universality Index and evaluates whether directions transfer across model architectures.
14. Steering Vector Factory
Creates inference-time steering vectors from refusal directions for reversible experimentation.
15. Evaluation Suite
Computes refusal rate, perplexity, coherence, KL divergence, CKA, and effective rank.
Analysis-Informed Pipeline
The informed workflow connects analysis to modification:
SUMMON -> Load model
PROBE -> Collect activations
ANALYZE -> Map geometry
DISTILL -> Extract directions using analysis results
EXCISE -> Project out selected directions
VERIFY -> Check for self-repair and compensate if needed
REBIRTH -> Save the model and analysis metadata
During ANALYZE, the pipeline uses several modules to configure later stages:
| Analysis module | Detects | Configures |
|---|---|---|
| Alignment Imprint | DPO, RLHF, CAI, or SFT signatures | Regularization and projection strength |
| Concept Cone Geometry | Polyhedral or linear refusal structure | Number of directions, from 1–8 |
| Cross-Layer Alignment | Direction clusters and persistence | Layer selection |
| Defense Robustness | Self-repair risk and entanglement | Refinement passes and skipped layers |
Additional Techniques
The toolkit also describes the following techniques:
| Technique | Description |
|---|---|
| Expert-Granular Abliteration (EGA) | Decomposes refusal signals into per-expert components for MoE models |
| Chain-of-thought-aware ablation | Attempts to separate refusal directions from reasoning-critical directions |
| COSMIC layer selection | Selects layers based on cosine similarity between harmful and harmless representations |
| Parametric kernel optimization | Uses bell-curve layer weighting with seven global parameters and Optuna TPE search |
| Refusal Direction Optimization (RDO) | Refines SVD-extracted directions with gradient-based optimization |
| Float direction interpolation | Interpolates between SVD direction indices with Gaussian weighting |
| KL-divergence co-optimization | Uses post-projection feedback to revert over-projected layers |
| Component-specific scaling | Applies separate projection strengths to attention and MLP components |
| LoRA-based reversible ablation | Uses rank-1 LoRA adapters instead of permanent weight surgery |
| Activation winsorization | Clamps activation vectors to a percentile range before SVD |
Telemetry-enabled runs contribute data to the project’s crowd-sourced research platform.
Reversible vs. Permanent Methods
OBLITERATUS supports two broad intervention strategies:
- Permanent weight projection
- Reversible inference-time steering
Weight Projection
Weight projection modifies the model directly:
obliteratus obliterate meta-llama/Llama-3.1-8B-Instruct \
--method advanced
Advantages:
- No additional runtime hooks
- Works with standard inference engines
- Suitable for one-time conversion
- Can provide a thorough intervention
Tradeoffs:
- Keep backups because the change is permanent
- Changing the configuration requires another run
- Model license terms may affect redistribution or commercial use
Use this approach only when you have validated the output and understand the model’s licensing requirements.
Steering Vectors
Steering vectors apply an intervention during inference without changing model weights:
from obliteratus.analysis import (
SteeringVectorFactory,
SteeringHookManager,
)
from obliteratus.analysis.steering_vectors import SteeringConfig
# Build a vector from an extracted refusal direction
vec = SteeringVectorFactory.from_refusal_direction(
refusal_dir,
alpha=-1.0,
)
# Or build one from contrastive activation pairs
vec = SteeringVectorFactory.from_contrastive_pairs(
harmful_acts,
harmless_acts,
)
config = SteeringConfig(
vectors=[vec],
target_layers=[10, 11, 12, 13, 14, 15],
)
manager = SteeringHookManager()
manager.install(model, config)
output = model.generate(input_ids)
# Restore the model's normal behavior
manager.remove()
Advantages:
- Reversible
- Supports an adjustable
alpha - Allows multiple vectors to be composed
- Does not modify model weights
- Useful for per-session or per-user experiments
Tradeoffs:
- Requires inference-time hook infrastructure
- Adds runtime overhead
- May be less thorough than weight projection
Choosing an Approach
| Use case | Recommended approach |
|---|---|
| Production API | Permanent weight projection after validation |
| Research experiments | Reversible steering vectors |
| Red teaming | Steering vectors with adjustable alpha
|
| Creative writing | Weight projection with the advanced method |
| Security testing | Weight projection with the aggressive method |
| Multi-tenant systems | Steering vectors per user or session |
Real-World Use Cases
1. API Testing and Development
When building content-generation APIs, test data should cover edge cases that the production system may encounter. An aligned test model may refuse to generate some of those cases.
A development team using OBLITERATUS for test-data generation reported that the modified model produced broader coverage of morally complex and borderline scenarios. The resulting test suite helped expose issues that might otherwise have reached production.
For API developers, the practical pattern is:
- Keep the unrestricted model in a development-only environment.
- Generate a broad test corpus.
- Run the corpus through the production API.
- Verify moderation, validation, and error-handling behavior.
- Never expose the modified model directly to end users without safeguards.
Apidog users building API testing pipelines can use a modified local model as one component of a broader test-generation workflow.
2. Academic Research
Researchers studying model behavior can use the analysis modules to compare refusal geometry across models. The project’s telemetry dataset is intended to support benchmark collection across different architectures and configurations.
3. Creative Writing Applications
Writers and game developers may encounter refusals when generating villain dialogue, morally ambiguous quests, or conflict scenarios. A modified model can be evaluated for these use cases while application-level controls remain in place.
4. Security Red Teaming
Security researchers can use controlled model modifications to study potential behaviors before reporting vulnerabilities. Keep testing isolated and follow responsible disclosure practices.
5. Localization and Multilingual Applications
Refusal behavior may vary by language. Testing the same prompts across languages can reveal inconsistent behavior and help teams decide whether to enforce policy in the model, the application, or both.
Alternatives and Comparisons
| Capability | OBLITERATUS | TransformerLens | Heretic | FailSpy abliterator | RepEng |
|---|---|---|---|---|---|
| Refusal direction extraction | Diff-in-means, SVD, and whitened SVD | Manual hooks | Diff-in-means | Diff-in-means | Diff-in-means |
| Weight projection | Seven presets with norm preservation | Not applicable | Bayesian-optimized | Basic | Not applicable |
| Steering vectors | Factory and hook manager | Not applicable | Not applicable | Not applicable | Core feature |
| Concept geometry analysis | Cones and solid angles | Not applicable | Not applicable | Not applicable | Not applicable |
| Alignment fingerprinting | DPO, RLHF, CAI, and SFT | Not applicable | Not applicable | Not applicable | Not applicable |
| Cross-model transfer analysis | Universality Index | Not applicable | Not applicable | Not applicable | Not applicable |
| Defense robustness evaluation | Ouroboros-effect analysis | Not applicable | Not applicable | Not applicable | Not applicable |
| Analysis-informed pipeline | Closed-loop feedback | Not applicable | Not applicable | Not applicable | Not applicable |
| Test coverage | 837 tests | Community-dependent | Unknown | None listed | Minimal |
| Model compatibility | HuggingFace models | Approximately 50 architectures | 16 tested | TransformerLens only | HuggingFace |
Choose an alternative when:
- TransformerLens is a better fit for general mechanistic interpretability research.
- SAELens is needed for sparse autoencoder analysis.
- RepEng is sufficient for basic steering-vector experiments.
OBLITERATUS is most relevant when you need:
- Refusal-specific analysis and intervention
- A verification-oriented pipeline
- A web interface for non-technical users
- Reversible steering or permanent projection
- Cross-model and alignment-geometry analysis
Conclusion
OBLITERATUS combines refusal-direction extraction, model analysis, weight projection, steering vectors, and post-run verification into one workflow for open-weight language models.
Its core workflow is:
- Select an open-weight model.
- Analyze refusal-related activations.
- Extract candidate directions.
- Apply a reversible or permanent intervention.
- Compare the original and modified models.
- Measure refusal rate, perplexity, coherence, and behavioral drift.
- Add application-level safeguards before deployment.
Use the least invasive method that meets your testing goal. Start with a small model and the advanced preset, preserve the original weights, and validate the output with a representative evaluation set.
Next steps:
- Try the HuggingFace Space for zero-setup testing.
- Install locally for GPU access and repeatable runs.
- Explore the analysis modules before modifying larger models.
- Use steering vectors when you need reversible experiments.
- Enable telemetry if you want to contribute anonymous benchmark data.
- Integrate modified models only into controlled development or research workflows.
FAQ
Is OBLITERATUS legal to use?
OBLITERATUS is open-source software released under the AGPL-3.0 license. You must have the right to use and modify the models involved and comply with their individual licenses. Commercial users should review the AGPL requirements and obtain a commercial license if needed.
Will it work on closed-source models such as GPT-4?
No. OBLITERATUS requires access to model weights and internal activations. Closed-source APIs do not expose the parameters required for abliteration.
Does removing refusal behavior make models dangerous?
It can remove an important safety layer, so treat modified models as higher-risk components. Use isolated environments, evaluate outputs, enforce application-level policy, and avoid exposing an unreviewed model directly to users.
How long does the process take?
The stated runtime is approximately 10–30 minutes, depending on model size and hardware. Models below 8B parameters may complete in roughly 10–15 minutes, while larger models can take 30 minutes or more.
Do I need a GPU?
No local GPU is required when using HuggingFace Spaces. For local runs, a GPU significantly improves performance. CPU execution is practical primarily for tiny models.
Can I reverse the changes?
Permanent weight projection is not reversible, so keep a backup of the original model. Steering vectors are reversible because they are applied at inference time and can be removed.
Will the model still follow instructions?
The goal is to target refusal-related directions while preserving instruction-following behavior. Verify this with A/B tests and quality metrics for your specific model.
What models are supported?
The project lists 116 curated models across five compute tiers, from GPT-2 to models such as DeepSeek-V3.2 685B. It is designed for HuggingFace transformer models, including Llama, Mistral, Qwen, Gemma, Phi, and others.
How can I contribute to the research?
Enable telemetry with the CLI option:
obliteratus obliterate MODEL_NAME --contribute
You can also set:
export OBLITERATUS_TELEMETRY=1
Telemetry-enabled benchmark data contributes to the project’s community dataset and public leaderboard.
Top comments (0)