Zuqi Zhang, Chenghe Sun, Xiangyi Chu, Wei-Han Yu, Ka-Fai Un, Rui Martins, Pui-In Mak, Jiawei Xu — Forty-third International Conference on Machine Learning (ICML 2026). Verified via the ICML 2026 poster listing (icml.cc/virtual/2026/poster/62811). Anonymous code release: anonymous.4open.science/r/STLA.

Core Contribution

Post-training quantization (PTQ) for LLMs has two dominant rounding styles that the field usually treats as alternatives: learning-based adaptive rounding (accurate, expensive) and compensation-based rounding (fast, cheaper). STLA’s claim is that the reason hybrids underperform is not “you can’t combine them,” but spatiotemporal misalignment when the two phases are decoupled — temporal parameter conflict, Round-to-Nearest (RTN) assumptions that break once learning moves weights, and spatially inconsistent local objectives that don’t know what later compensation will do. The contribution is a rounding-optimized PTQ framework that (1) collocates learning and compensation inside clusters and (2) gives the local solver a global lookahead objective derived from the Schur Complement so current rounding anticipates future weight compensation.

Method

Problem setup (what is being optimized)

PTQ here means: take a trained full-precision LLM, choose low-bit weight representations with no (or minimal) end-to-end fine-tuning, and decide per-weight (or per-block) rounding so reconstruction error of layer outputs stays small. Adaptive rounding methods treat the continuous→discrete map as an optimization variable. Compensation methods adjust remaining weights to cancel earlier quantization error. STLA targets the interaction between those two.

The paper names three failure modes of the “decoupled” paradigm:

  1. Temporal parameter conflict — learning-phase parameters and compensation-phase parameters fight when run as separate stages.
  2. Invalidated RTN assumption — once learning moves away from nearest-neighbor rounding, later stages that still assume RTN-initialized structure become mis-specified.
  3. Spatially inconsistent objectives — optimizing rounding for one weight group without a global view of how later groups will compensate produces locally good, globally bad solutions.

Methodology

1. Cluster-wise integrated rounding optimization (temporal fix).
Instead of “learn all roundings, then compensate,” STLA partitions weights into clusters and runs learning + compensation together inside each cluster. That collocates the two phases in time so the compensation signal is available while rounding decisions are still being made, rather than only afterward.

2. Hessian-guided clustering (what defines a cluster).
Clustering is not arbitrary blocking. STLA uses Hessian information — both diagonal and off-diagonal structure — to group weights so that intra-cluster error cancellation is maximized. Intuition: if two weights’ quantization errors are correlated in the second-order loss landscape, putting them in the same integrated solve lets compensation cancel what rounding introduces. (Exact clustering algorithm, cluster count schedule, and per-layer vs global clustering are PDF-level details I have not yet independently confirmed from full text.)

3. Schur-Complement lookahead / spatial alignment (global objective).
For the spatial problem — local rounding that ignores future compensation — STLA derives a unified global objective from the Schur Complement. In block-matrix terms, the Schur Complement is the tool that expresses “the effect of optimizing block A after accounting for the optimal response of block B.” Applied here: the solver for the current cluster’s rounding is asked to look ahead to the optimal future compensation of remaining weights, so local decisions are aligned with that future optimum rather than with a myopic reconstruction loss.

4. Efficiency claim.
The pitch is SOTA low-bit PTQ without giving up the speed advantage of compensation-style methods — i.e., not “just run OmniQuant longer.”

Dataset / evaluation protocol

The ICML abstract claims “extensive experiments” establishing SOTA low-bit PTQ, but the public poster abstract does not enumerate models, bit-widths, calibration corpora, or task suites. Pending a full-text pass, treat the following as standard PTQ evaluation practice this paper is almost certainly in conversation with, not as confirmed STLA experimental choices:

  • Calibration data in this literature is typically a small held-out text sample (often WikiText-2 / C4 snippets, tens to low hundreds of sequences) used only to estimate Hessians / activation statistics — not training data.
  • Metrics are typically WikiText-2 / C4 perplexity plus zero-shot accuracy on reasoning suites (ARC, HellaSwag, PIQA, WinoGrande, etc.).
  • Models in contemporary LLM-PTQ papers are usually LLaMA-family (and often Qwen) across 7B–70B.

Until the PDF is grounded, I am not claiming specific PPL deltas, bit-widths (W3/W4/…), or model list for STLA. What is confirmed from the venue abstract: the method is rounding-optimized PTQ; the novelty is spatiotemporal alignment via cluster-integrated rounding + Schur lookahead + Hessian-guided clustering; code was released anonymously for review.

Limitations

  • Capability-only evaluation. Like almost every PTQ paper in this batch, the objective is accuracy/perplexity after compression. Refusal behavior, jailbreak ASR, and multilingual safety are out of scope — which is exactly why it’s useful as a baseline method to re-run under a safety probe, not as a safety result.
  • Full experimental tables (models, bits, calibration set size, wall-clock) still need PDF grounding; abstract-only notes can’t support numeric claims.
  • Hessian-guided clustering + Schur objectives can be computationally nontrivial at 70B scale; the paper claims high efficiency, but the cost profile vs GPTQ/AWQ/OmniQuant needs the results section.

Relevance to My Niche

High for Direction 2. STLA is a current, ICML-accepted SOTA-style PTQ scheme — the kind of compressor any credible “quantize-then-probe refusal” study has to include alongside older baselines (GPTQ, AWQ, RTN, GGUF paths). It does not touch safety; it widens the menu of production-plausible compressors whose effect on the refusal signal nobody has measured. Pairs with the Week 1 Egashira line (quantization as adversarial trigger) as the complementary question: even honest STLA-style PTQ may shift safety unevenly; adversarial PTQ is a different threat model entirely.