Shen Li, Liuyi Yao, Lan Zhang, Yaliang Li: Safety Layers in Aligned Large Language Models: The Key to LLM Security, ICLR 2025 (OpenReview; proceedings PDF; arXiv:2408.17003). Affiliations: USTC; Hefei Institute of Artificial Intelligence. Code: listen0425/Safety-Layers.
Core Contribution
Aligned chat models refuse harmful prompts, but fine-tuning (even on mostly benign data) can wipe that behavior. Refusal concentrates in a contiguous band of middle layers (“safety layers”), where last-token hidden states start to separate normal vs malicious queries. They localize that band, then propose Safely Partial-Parameter Fine-Tuning (SPPFT): freeze gradients on the safety layers and update the rest, so downstream fine-tuning hurts security less than full fine-tuning.
Method (as published)
Models (all 32 layers except Gemma’s 18). Llama-3-8B-Instruct (32L), Llama-2-7B-Chat (32L), gemma-2b-it (18L), Phi-3-mini-4k-instruct (32L), plus pretrain counterparts for the existence check.
Existence check. Same chat template for all prompts. At each layer, take the last-position residual vector. Over P=100 normal and Q=100 malicious queries with r=500 random pairings, compare cosine similarity for normal-normal, malicious-malicious, and normal-malicious pairs. Aligned models show a mid-depth onset where N-M diverges from N-N; pretrained siblings do not, so the band is attributed to alignment, not architecture alone.
Localization. Start from the divergence window, then refine bounds with a progressive scaling algorithm: multiply candidate layers’ weights by a factor \(\alpha\) and read the over-rejection count on an over-rejection dataset \(D_o\) (721 borderline-safe problems); the true safety band is where scaling most changes rejection behavior. Located ranges (Table 1):
| Model | Safety layer range |
|---|---|
| Llama-3-8B-Instruct | [7, 12] |
| Llama-2-7B-Chat | [9, 14] |
| gemma-2b-it | [8, 11] |
| Phi-3-mini-4k-instruct | [11, 15] |
SPPFT. During fine-tuning, fix gradients of the localized safety layers; train the rest. Fine-tuning datasets: a normal set \(D_N\) (thousands of alpaca-finance entries), an implicit-attack set \(D_I\) (benign-looking instructions, outputs start positively), a backdoor set \(D_B\), and harmful sets \(D_H\) (1,000 benign + 1,000·p malicious, p ∈ {0.05, 0.1, 0.2}). Baselines: full fine-tuning (FullFT), neuron-freezing (NFFT) on Wei-style security-critical neurons, and LISA (harmful-attack setting).
Results
- Normal-data fine-tuning still erodes safety, and SPPFT blunts it. Harmful-response rate SPPFT vs FullFT: Llama-3 9.62% vs 44.42%, Phi-3 7.12% vs 40.00%, Gemma 5.58% vs 18.27%, Llama-2 2.88% vs 10.58%.
- Under harmful-data attack (p=0.1): Llama-2 SPPFT 25.6% vs FullFT 53.7% vs LISA 41.4%; Phi-3 SPPFT 23.3% vs FullFT 90.7% vs LISA 61.3%.
- Task performance preserved: Rouge-L differences vs FullFT typically <0.01, MMLU stable.
- Freezing non-safety layers does not protect security the same way (localization matters); pretrained models lack the N-M divergence signature.
Limitations / gaps for my niche
- English, full-precision evaluation. No MultiJail / resource-tier languages. No INT4 / GGUF / PTQ stress test of the frozen safety band.
- Intervention is mostly preserve safety under capability FT, not install new safety via single-layer RL (contrast Zhang capability GRPO; contrast ESI SET below).
- Over-rejection is both a localization tool and a failure mode; SPPFT’s over-refusal tradeoff needs careful reading in the tables.
Relevance
Related work for Dipen’s “middle layers = safety” pitch: Tier-1 evidence that refusal is layer-localized. Does not close quantization × multilingual ASR, and does not test whether localized training of safety survives compression across languages. I would cite it as prior art that middle-layer safety exists; the quant × multilingual × ASR stress test stays open for me.