Detecting the Semantic Fixed Point

Jiawei Gu, Ziyue Qiao, Xiao Luo — Forty-third International Conference on Machine Learning (ICML 2026), Oral (poster session also listed). Verified via the ICML 2026 oral/poster pages (icml.cc/virtual/2026/oral/71056, …/poster/65483) and co-author listing (Qiao homepage notes Oral, ~0.7% acceptance).

Core Contribution

Early-exit methods usually ask: “is the output confident enough to stop?” This paper asks a different question: “has the hidden state stopped changing in a meaningful way?” Framing each Transformer layer as refining a representation toward a fixed point, they show layer-wise updates have a sharp two-phase geometry — large volatile early updates, then small directionally aligned updates that mostly propagate an already-formed representation — and turn that into a training-free exit rule with \(\mathcal{O}(d)\) overhead per layer and no architectural changes.

Method

Conceptual model

Write \(h^{(\ell)}\) for the hidden state after layer \(\ell\). The layer update is \(\Delta^{(\ell)} = h^{(\ell)} - h^{(\ell-1)}\) (or an equivalent residual-form difference; exact definition needs the PDF). Full-depth inference runs all \(L\) layers. Early exit at \(\ell^* < L\) skips the remaining blocks and reads out from \(h^{(\ell^*)}\) via the existing unembedding / task head — no learned early-exit heads.

Existing methods proxy “done” with output confidence (entropy of the next-token distribution, margin, etc.). That couples the exit decision to a vocabulary-sized softmax and to a signal that can be confident and wrong, or still moving internally while the top token looks stable.

Empirical geometric observation

Across layers, update trajectories show:

1. Phase 1 (early layers): large, volatile \(\Delta^{(\ell)}\) — the representation is still being formed.
2. Phase 2 (later layers): small updates that are directionally aligned with each other — the model is mostly propagating an already-formed semantic state.
3. The phase transition is sharp, which is what makes a simple threshold rule viable.

Exit criterion

At each layer \(\ell\), track two scalar signals:

1. Normalized update norm — a scale-free measure of step size \(\|\Delta^{(\ell)}\|\) (normalized, e.g. by \(\|h\|\) or a running baseline; exact normalization in the PDF). “Step size vanishes” means the representation has stopped moving much.
2. Cosine similarity between consecutive updates — \(\cos(\Delta^{(\ell)}, \Delta^{(\ell-1)})\). “Direction stabilizes” means successive refinements point the same way rather than thrashing.

Exit when both indicate convergence (step small and direction stable). That conjunction is the method: either signal alone would be insufficient (small noisy jitter can be tiny but misaligned; large aligned steps still mean “not done”).

Systems properties

• Overhead \(\mathcal{O}(d)\) per layer — norm and cosine over the hidden dimension; independent of vocabulary size (no extra LM-head pass for the exit decision).
• No learned components — no exit classifiers, no early-exit fine-tuning, no added parameters.
• No architectural modifications — drop-in at inference on a stock Transformer stack.

Dataset / evaluation protocol

Confirmed from the ICML abstract:

The abstract does not list the exact QA/commonsense suite (BoolQ / ARC / HellaSwag / PIQA / etc. are the usual suspects in this literature). Pending PDF: per-task tables, threshold selection (fixed vs calibrated), token-level vs sequence-level exit, and comparison set (confidence-based early exit, DeeBERT-style, LayerSkip, etc.).

What “dataset” means here: this is not a new benchmark release. The “data” are standard downstream eval sets used as accuracy anchors while measuring compute (FLOPs / layers executed). Calibration of exit thresholds — if any — would typically use a small held-out slice of the same task distribution; confirm in PDF whether thresholds are task-agnostic or tuned per task.

Limitations

• Efficiency paper, not a safety paper: no refusal, jailbreak, or robustness evaluation under early exit. If the refusal signal is low-dimensional and late-forming, exiting on geometric convergence could systematically skip safety-critical layers — an untested interaction that matters for my niche.
• LLaMA-2 7B/13B only in the abstract; transfer to LLaMA-3 / MoE / instruction-tuned chat models is unconfirmed here.
• Training-free is a strength for deployment and a limit on adaptation: no learned exit policy for domains where the two-phase geometry is weaker (e.g. long CoT).

Relevance to My Niche

Adjacent but sharp for Direction 2. Early exit is a second compression mechanism beside quantization: less compute by skipping layers rather than lowering bit-width. Same open safety question as PTQ — does the efficiency trick move the refusal boundary? STLA compresses weights; this compresses depth. A quantize-then-probe study that also ablates early-exit would be asking whether two independently “capability-preserving” compressors are safety-preserving when stacked. Methodologically, the paper is also a reminder that confidence ≠ internal convergence — relevant to judge/refusal metrics that trust surface fluency.