Common Failure Modes of Quantization Techniques Across the Survey

Context

Quantization is how large language models get smaller so they can run on ordinary hardware. Weights that were stored at high precision are stored at fewer bits, which cuts memory and cost. Papers that study this almost always ask whether the compressed model still answers the same questions at the same level of quality. Much less often do they ask whether it still refuses harmful requests, and almost never whether that refusal holds in languages other than English.

For my literature review, I read through top papers in this area of research, combined survey, and analyzed across eight quantization-related entries (GPTQ, AWQ, SmoothQuant, LLM.int8(), NF4, AQLM, QLoRA, GGUF, and early exit as a second compression axis) to see whether each method fails in its own way or whether the failures cluster. Most of these papers follow from the design choices around the tools that decide which weights to protect or how to round them to optimize for reconstruction error and perplexity, which are capability scores. Refusal and fairness live in a largely different set of weights.

What I’ve produced here is a short list of recurring gaps across these papers to understand general patterns of disadvantages in quantization, rather than a long list of idiosyncratic bugs. Safety work is done on research quantizers, but those who ship (GGUF) have no safety measurement in any language. Static methods lean on a calibration set, and that set’s content dominates measured attack success, yet no paper varies calibration language.

Capability-based saliency leaves refusal under-protected by construction, which also explains why LLM.int8() looks oddly stable: it keeps outlier channels in high precision for capability reasons and thereby accidentally protects some of the weights a safety-aware method would try to save. The common threat model in the literature is either a sophisticated attacker who controls the full-precision checkpoint or no attacker at all; the ordinary case (a reputable checkpoint, quantized locally with a stock tool, degrading unevenly by language) sits between those poles and is mostly unmeasured.

Finally, evaluation itself fails where it is most needed: quantization hurts fluency, and fluency drops hardest in low-resource and non-Latin settings. LLM judges confuse fluency with both refusal and compliance, so the quantized × low-resource cell is where automatic scores are least trustworthy. Terms: primer. Study design this feeds: research log.

Scope

Eight of the 25 entries touch quantization directly: #4 Exploiting, #5 Mind the Gap, #6 Marchisio, #7 Al Hakim, #8 Q-resafe, #12 STLA, plus #13 Early Exit as a second compression axis. Three more (#9 Safety Layers, #10 ESI, #11 Arditi) localize safety and explicitly decline to remeasure under compression, which is itself one of the findings below.

Technique inventory

The Family and Criticality criterion columns are background knowledge about how each quantizer works, not findings from the surveyed papers. None of the notes under content/notepad/ describes GPTQ’s, AWQ’s, SmoothQuant’s or LLM.int8()’s saliency criterion, and imatrix appears in no surveyed paper. Verify that column against each method’s own paper before it enters a methods section. The Appears in and Recorded failure columns are sourced and traceable.

Six recurring failures

1. Measurement runs on research recipes; deployment runs on GGUF

One entry in 25 touches the format practitioners run, and it uses GGUF as an attack target rather than measuring degradation under it. Every other quantization arm in the table is GPTQ, AWQ, SmoothQuant, LLM.int8(), FP8, NF4, AQLM, LLM-QAT or QLoRA. #4 and #7 both list the absence of GGUF as an explicit gap.

This is not a naming difference, though the surveyed record establishes less about how GGUF differs than I would like. #5 confirms only “per-block scales, mixed precision,” at abstract level. My own understanding, unsourced here, is that GGUF also quantizes the block scales themselves and varies bit-width by tensor type, which would mean “GPTQ at 4 bits” does not extrapolate to Q4_K_M. That needs checking against llama.cpp before it carries weight. What the record does support is the weaker and still sufficient point: no surveyed paper measures safety under GGUF, so the extrapolation is untested either way. H4 in the research log treats directional agreement between the two as a hypothesis, which is the right posture.

2. Calibration is the controlling variable, and nobody varies its language

Every static method in the table consumes a calibration set. Q-resafe is the only entry that treats calibration content as an experimental factor, and the effect dominates every other variable it measures: on Llama INT4, QLoRA moves from 42.3 ASR under benign calibration to 85.3 under directly harmful, and indirect harm (83.4) costs nearly as much as direct. Even benign calibration degrades safety relative to FP16.

Al Hakim’s finding that dynamic PTQ is more stable than static is consistent with this: FP8 and LLM.int8() take no calibration set, so one damage channel is absent. Worth flagging as a confound in their grid, since the dynamic arm is also the 8-bit arm, so “dynamic is safer” and “8 bits is safer” are not separated there.

What no paper does: vary calibration language. Q-resafe varies harmfulness in English throughout. Al Hakim’s criticality scoring uses StereoSet and AdvBench, English-only, then applies the resulting mask to multilingual evaluation. This is the cleanest unexploited factor in the table. It may be sharper under GGUF than anywhere else, if the imatrix path does what I think it does and bakes a calibration-derived importance prior into the shipped artifact. No surveyed paper says so: imatrix appears in none of them, and that claim rests on my reading of llama.cpp practice rather than on anything in this survey. The factor is worth testing regardless of how that check comes out, since the calibration sets in Q-resafe and Al Hakim are English either way.

3. Saliency criteria track perplexity, so refusal is unprotected by construction

Every criticality criterion in the inventory above is a capability proxy: reconstruction error, activation magnitude, outlier detection. Three papers in the survey exist because that criterion fails for safety. Al Hakim replaces it with FAIRSCORE + SAFESCORE, Q-resafe restricts repair to a SNIP mask, ESI replaces it with |σ·∇S|.

ESI quantifies how badly the capability criterion misidentifies safety weights: perturbing the top 1% by ESI raises HarmBench ASR from 15.3 to 59.1, while SNIP, Wanda, SN and GMT reach at most 37.6 on the same budget. The safety-critical set and the capability-critical set are largely different sets.

Compose that with Arditi: if refusal is carried by a single residual-stream direction, it contributes almost nothing to reconstruction error. A quantizer minimizing reconstruction error will therefore spend its error budget in the subspace that carries refusal. That predicts safety degrading faster than capability under any perplexity-calibrated quantizer, which is what Q-resafe observes: LLM-QAT on Llama-2-7B-Chat stays within 2% of FP16 on utility while INT4 ASR under benign calibration reaches 16.9 against a 0.3 FP16 baseline. Capability holds and safety does not, in the same run. It is a mechanism, not a correlation, and it is testable per layer.

4. LLM.int8() is already accidental critical-weight protection

Its stability in Al Hakim’s grid needs no separate explanation. Retaining outlier channels in FP16 is Al Hakim’s mitigation with a capability scoring function substituted for a safety one, arrived at for unrelated reasons. This makes it the right control arm for the protection stretch goal: it isolates how much of AWQ-trust’s recovery comes from protecting anything versus protecting the right weights.

5. Two threat models, and the common case sits between them

#4 and #5 assume the attacker controls the full-precision checkpoint and the victim quantizes locally. Both require knowledge of the rounding scheme, both carry a supply-chain precondition, and neither proposes a defense. #7 and #8 assume no adversary at all, and Q-resafe’s own note flags its threat model as calibration-induced rather than adversarial.

Unoccupied between them: a benign checkpoint from a reputable publisher, quantized by the user with a stock tool, degrading unevenly by language, with nobody attacking anything. That is the modal deployment and it is the research log threat model.

6. Evaluation fails precisely at the quantized × low-resource cell

Five entries converge here and they compound. Marchisio: a 1.7% automatic drop in Japanese corresponds to 16.0% human-judged, roughly 10x underestimation, worst on non-Latin scripts. Al Hakim: single Gemini judge, no inter-judge reliability. ESI: single judge family, partly ablated. Beyer: the four-stage pipeline is too noisy for single-number comparison. Fassbender: fluent compliance masks incoherence.

Stacked, these describe one failure mode with a name. Quantization degrades fluency, degradation is worst in non-Latin and low-resource settings, and LLM judges conflate fluency with both refusal and compliance. A Q2_K model emitting degraded Swahili can be scored as refusal (false safe) or as compliance (false unsafe). No entry in the survey validates a judge under the joint condition. This is why H3 (fluency does not fully explain H1/H2) is load-bearing rather than a robustness check.

Secondary observations

Model coverage is narrow and at the wrong scale. #7 uses Gemma-7B, Llama-3.1-8B, Qwen-2.5-7B; #8 uses Llama-2-7B-Chat and Gemma-7B-Instruct; #13 uses LLaMA-2 only. Arditi is the sole entry spanning 1.8B to 72B. Quantization damage grows as models shrink and low-resource competence collapses as models shrink, so the interaction should be largest at 1B to 3B, which is the range GGUF exists to serve and the range nobody has measured.

Codebook methods may not accept a weight mask. AQLM reshapes weight geometry rather than rounding in place, so per-weight criticality masks (SNIP, ESI, refusal-direction projection) may not transfer. Scope this out of the protection stretch goal early rather than discovering it late.

STLA is a bridge, not just a baseline. Its Hessian-guided clustering is per-layer second-order sensitivity machinery pointed at a reconstruction objective. Substituting a safety score for that objective is close to the layer-resolved experiment, and it is the strongest available evidence that per-layer quantization error analysis is tractable.

Early exit splits compute from consumption. #13 flags that late-forming refusal could be skipped. The middle-layer literature complicates this: knowing where refusal is computed does not say where it is read. The same distinction applies to the layer-resolved analysis here.

Unverified

The GGUF mechanism claims above (super-block scale quantization, per-tensor bit assignment in Q4_K_M, imatrix calibration defaults) are not sourced from any surveyed paper. #5 confirms only “per-block scales, mixed precision” at abstract level. Verify against llama.cpp source before any of it enters a methods section. The specific claim worth checking first: whether Q4_K_M assigns higher precision to attention.wv and feed_forward.w2. If so, llama.cpp may already be partially protecting the middle V/MLP band identified by Safety Layers and ESI, for perplexity reasons and by accident. That assignment is uniform across depth, so it would not track a band, and either result is reportable.

Ranked Openings

1. No published safety measurement under GGUF, any language. The primary claim needs no novelty argument beyond stating this.
2. Calibration language is unvaried, despite Q-resafe showing calibration content dominates. The factorial design (tier × k-quant × calibration language) is unrun regardless. If the imatrix path works as I expect (unverified, see below), GGUF sharpens it further by baking the prior into the shipped file.
3. Capability saliency misses safety, with ESI’s 15.3 → 59.1 as published evidence and Arditi’s low-rank result as the mechanism. This predicts the H1 result rather than rationalizing it afterward.
4. No judge validated under quantized × low-resource conditions, which is the condition Marchisio says breaks measurement. Standalone methods contribution inside the main paper.
5. Everything is 7-8B while GGUF serves 1-3B.

On the open question in the primer (headline mechanistic claim, or supporting experiment): failure 6 argues for keeping \(\Delta_{\mathrm{HL}}\) × GGUF as the headline and refusal-direction localization as the mechanism section. The layer-resolved measurement depends on activation probing under direct control; the \(\Delta_{\mathrm{HL}}\) number depends on a judge that five surveyed entries call unreliable under exactly these conditions. Lead with the claim whose measurement is defensible and use the mechanism to explain it.

Research Thread to Pursue

Al Hakim’s criticality scoring is English-only, yet Llama-3.1 Korean %Safe recovers from ~9.6 to ~66.1 under AWQ-trust, and Arabic from ~24.7 to ~89.1. Two readings: safety-critical weights are largely language-shared, which supports the single-direction view, or the recovery is restored general competence in those languages and “%Safe” is riding on fluency. The second reading would mean a good part of this literature measures competence and reports alignment. Separating them needs a capability-matched control and is directly in scope for Phase A.