Kazuki Egashira, Robin Staab, Mark Vero, Jingxuan He, Martin Vechev — Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), ETH Zurich (SRI Lab). Verified via the ICML 2025 poster listing, the PMLR proceedings page (proceedings.mlr.press/v267/egashira25a.html), and OpenReview.

Core Contribution

Direct follow-up to Exploiting LLM Quantization (NeurIPS 2024, see companion entry). That paper attacked naive, simple quantization formats (LLM.int8, NF4, FP4). This one closes the obvious gap: GGUF, the block-wise quantization format that actually ships in the tools people use day-to-day — llama.cpp and ollama — was not covered by the original attack and was, going in, plausibly more resistant because its quantization scheme is more complex (per-block scales, mixed precision). The paper shows that added complexity is not a defense.

Method

The key technical move: exploit the quantization error — the gap between original full-precision weights and what you get after quantizing and then de-quantizing them. GGUF’s block-structured scheme still has slack in this error term, and the authors show it’s enough room to construct a full-precision model that both (a) reads as benign under direct full-precision inspection, and (b) reliably de-quantizes to a version carrying the injected malicious behavior. The attack trains the target malicious model while constraining its weights to stay within the quantization-error tolerance that maps back to the “clean-looking” full-precision checkpoint. This is a natural generalization of the constraint-projection idea from the NeurIPS 2024 paper, adapted to GGUF’s specific (de)quantization arithmetic.

Limitations

  • Same supply-chain precondition as the earlier paper: the victim has to download a full-precision checkpoint from an untrusted source and quantize it locally into GGUF themselves. It’s not a remote/inference-time attack.
  • I could not confirm from the pages I fetched exactly which base models (sizes, families) were used in evaluation, nor the specific attack-success-rate numbers — flagging that as unconfirmed pending a read of the actual PDF (available at the PMLR link above).
  • The paper’s own framing states the point plainly: “the complexity of quantization schemes alone is insufficient as a defense” — which is a limitation of the field’s assumptions more than of this paper, but worth noting it doesn’t propose a robust fix, just demonstrates the gap persists.

Relevance to My Niche

This is the more practically alarming of the two Egashira et al. papers for my niche, because GGUF is the format actually in circulation for locally-run, community-quantized models — exactly the deployment pattern (download a GGUF from Hugging Face, run it with ollama) that a lot of real-world “small/local LLM” usage follows. For red-teaming quantized models, this says the attack surface isn’t confined to research-lab quantization methods; it reaches the exact tooling practitioners use. Together with the NeurIPS 2024 paper, this gives me a two-paper empirical anchor that “quantization as adversarial trigger” is a validated, peer-reviewed line of work, not a one-off finding.