Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, Martin Vechev — Advances in Neural Information Processing Systems 37 (NeurIPS 2024), ETH Zurich (SRI Lab). Verified via the NeurIPS 2024 poster listing and the ETH SRI publication page.
Survey table
| Column | Entry |
|---|---|
| Specific authors | Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, Martin Vechev (ETH Zurich, SRI Lab) |
| Objectives | Demonstrate that quantization itself is an attack surface: craft a full-precision model that is genuinely benign under inspection and safety checks but becomes malicious only after the victim quantizes it locally |
| Methodology | Three-stage attack: (1) fine-tune a benign base into a malicious model; (2) compute the quantization pre-image, the constraint set of full-precision weights that all map to the same quantized model; (3) projected gradient descent back to benign full-precision behavior while staying inside that constraint set; targets LLM.int8(), NF4, and FP4 as shipped in Hugging Face transformers |
| Dataset | Three attack scenarios: vulnerable code generation, content injection, and over-refusal denial of service (per-scenario ASR percentages not independently confirmed in my note) |
| Open source | The SRI Lab group typically releases code, but I have not logged a repo link for this paper; verify on the NeurIPS/ETH SRI pages before citing availability |
| Advantages | First demonstration (as far as I could find) that compression is a deliberate adversarial trigger, not just incidental safety erosion; the many-to-one pre-image insight is elegant and general; reframes evaluation practice, since testing the full-precision checkpoint alone certifies nothing about its quantized descendants |
| Disadvantages | Assumes the attacker knows which zero-shot round-to-nearest quantization the victim will use, and does not cover complex schemes like GGUF (hence the ICML 2025 follow-up); requires the victim to download and locally quantize an untrusted checkpoint; feasibility proof with mitigation left open |
Core Contribution
This is, as far as I can find, the first paper to show that quantization itself is an attack surface — not just something that incidentally degrades safety. The threat model is the interesting part: an attacker distributes a full-precision model that is genuinely benign when you evaluate it, passes safety checks, looks clean on utility benchmarks — and then becomes malicious only once a downstream user quantizes it to run locally. The malice is dormant in the shared weights and is “developed” by the compression step the victim performs themselves.
Method
Three-stage attack:
- Inject. Fine-tune a benign base model on an adversarial task to produce a malicious model.
- Compute the quantization pre-image. Quantize the malicious model, then calculate the constraint set — the region of full-precision weight space that all maps to that same quantized model. Because quantization is many-to-one, a whole neighborhood of full-precision weights collapses to the identical low-bit model.
- Project back to “benign.” Use projected gradient descent to move the full-precision weights until they behave benignly at full precision, while staying inside the constraint set so they still quantize to the malicious model.
The attack targets the quantization methods shipped in Hugging Face transformers — LLM.int8(), NF4, and FP4. It’s demonstrated across three scenarios: vulnerable code generation, content injection, and an over-refusal (denial-of-service-style) attack. (I confirmed the three scenarios and the three quantization methods from the abstract and project page; I did not independently confirm the exact per-scenario attack-success percentages from the pages I read, so treat any specific number as unconfirmed until the PDF is checked.)
Limitations
- The original attack assumes the attacker knows which zero-shot / round-to-nearest style quantization the victim will use; it does not automatically generalize to every scheme. The same group’s follow-up (Mind the Gap, ICML 2025) exists precisely because more complex schemes like GGUF weren’t covered here.
- It requires the victim to download and locally quantize an attacker-supplied checkpoint — a real but specific supply-chain assumption (Hugging Face-style model sharing).
- It’s an existence/feasibility proof of the vulnerability, not a defense; mitigation is left largely open.
Relevance to My Niche
This is close to the center of the quantized-models half of my niche, and it reframes the whole question. Most “does compression hurt safety” work asks whether quantization accidentally erodes alignment. This paper says the more dangerous case is adversarial: quantization as a deliberate trigger. For red-teaming quantized models, that means the test can’t just be “quantize an honest model and re-run the safety eval” — you also have to reason about whether the distributor engineered the gap between full-precision and quantized behavior. Pairs directly with Mind the Gap (the GGUF extension) as a two-paper backbone for the quantization thread.