Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, Martin Vechev — Advances in Neural Information Processing Systems 37 (NeurIPS 2024), ETH Zurich (SRI Lab). Verified via the NeurIPS 2024 poster listing and the ETH SRI publication page.

Core Contribution

This is, as far as I can find, the first paper to show that quantization itself is an attack surface — not just something that incidentally degrades safety. The threat model is the interesting part: an attacker distributes a full-precision model that is genuinely benign when you evaluate it, passes safety checks, looks clean on utility benchmarks — and then becomes malicious only once a downstream user quantizes it to run locally. The malice is dormant in the shared weights and is “developed” by the compression step the victim performs themselves.

Method

Three-stage attack:

  1. Inject. Fine-tune a benign base model on an adversarial task to produce a malicious model.
  2. Compute the quantization pre-image. Quantize the malicious model, then calculate the constraint set — the region of full-precision weight space that all maps to that same quantized model. Because quantization is many-to-one, a whole neighborhood of full-precision weights collapses to the identical low-bit model.
  3. Project back to “benign.” Use projected gradient descent to move the full-precision weights until they behave benignly at full precision, while staying inside the constraint set so they still quantize to the malicious model.

The attack targets the quantization methods shipped in Hugging Face transformers — LLM.int8(), NF4, and FP4. It’s demonstrated across three scenarios: vulnerable code generation, content injection, and an over-refusal (denial-of-service-style) attack. (I confirmed the three scenarios and the three quantization methods from the abstract and project page; I did not independently confirm the exact per-scenario attack-success percentages from the pages I read, so treat any specific number as unconfirmed until the PDF is checked.)

Limitations

  • The original attack assumes the attacker knows which zero-shot / round-to-nearest style quantization the victim will use; it does not automatically generalize to every scheme. The same group’s follow-up (Mind the Gap, ICML 2025) exists precisely because more complex schemes like GGUF weren’t covered here.
  • It requires the victim to download and locally quantize an attacker-supplied checkpoint — a real but specific supply-chain assumption (Hugging Face-style model sharing).
  • It’s an existence/feasibility proof of the vulnerability, not a defense; mitigation is left largely open.

Relevance to My Niche

This is close to the center of the quantized-models half of my niche, and it reframes the whole question. Most “does compression hurt safety” work asks whether quantization accidentally erodes alignment. This paper says the more dangerous case is adversarial: quantization as a deliberate trigger. For red-teaming quantized models, that means the test can’t just be “quantize an honest model and re-run the safety eval” — you also have to reason about whether the distributor engineered the gap between full-precision and quantized behavior. Pairs directly with Mind the Gap (the GGUF extension) as a two-paper backbone for the quantization thread.