Tim Beyer, Sophie Xhonneux, Simon Geisler, Gauthier Gidel, Leo Schwinn, Stephan Günnemann — full title: “Position: LLM-Safety Evaluations Lack Robustness,” Forty-third International Conference on Machine Learning (ICML 2026), Position Paper track. Verified via the paper’s arXiv listing (arXiv:2503.02574, revised May 2026) and multiple independent citations confirming ICML 2026 acceptance. Full author list (six, including Stephan Günnemann, not captured by the original note’s “et al.”) confirmed from the arXiv metadata.

Core Contribution

An ICML Position paper — an argued stance rather than a new method or benchmark. The claim: current LLM safety-alignment research is hindered by “many intertwined sources of noise” — small datasets, methodological inconsistencies, and unreliable evaluation setups — to the point that it can be impossible to fairly compare attacks and defenses across papers, which slows genuine progress even as the leaderboard-style literature grows. The contribution is a systematic diagnosis of where that noise enters the pipeline, not a fix.

Method

Not an empirical study in the “here’s a new benchmark” sense — a systematic critique structured around four stages of the LLM safety-evaluation pipeline, with issues and practical impact identified at each:

  1. Dataset curation — small, often idiosyncratically constructed harmful-prompt datasets create reliability problems for comparing results across papers; what counts as a representative or sufficient harm-category sample is inconsistent field-wide.
  2. Automated red-teaming optimization — methodological inconsistencies in how attack-search methods are tuned and reported (e.g., search budget, hyperparameters, stopping criteria) make attack-strength claims hard to compare fairly between papers.
  3. Response generation — inconsistent generation procedures (sampling temperature, number of attempts, refusal-detection heuristics) affect reproducibility of “attack success” numbers across labs.
  4. Response evaluation / LLM-as-judge — reliance on LLM judges to grade whether a response was “harmful” introduces its own unreliability, since judge models have their own biases, inconsistencies, and sensitivity to prompt phrasing.

The authors propose a set of guidelines aimed at reducing noise and bias at each stage for future attack/defense papers, while explicitly acknowledging practical constraints (cost, dataset availability) behind why the field ended up here.

Limitations

  • Position papers set an agenda; they don’t close it. This paper diagnoses the problem precisely but doesn’t hand the field a ready-made fixed benchmark or evaluation protocol to adopt wholesale — the proposed guidelines are recommendations, not a new standard that’s already been validated at scale.
  • Being a critique of evaluation methodology rather than a new attack or defense, it can’t be used as a technique to build on directly — it’s a rigor bar to hold other work (including my own) against, not a method contribution.
  • The four-stage framing is comprehensive but general-purpose — it doesn’t specifically analyze whether the noise sources it identifies (small datasets, judge unreliability, etc.) are worse or better for non-English/multilingual safety evaluation specifically, which is exactly the gap most relevant to my own niche and is left unaddressed here.

Relevance to My Niche

High and directly actionable as a methodological rigor bar, if not as a subject-matter match. This paper hands me independent, ICML-2026-level cover for a claim I need in my own work: that benchmark-only, single-number safety claims are increasingly seen as insufficiently rigorous by the field itself, and that the specific failure points (small/idiosyncratic datasets, red-teaming methodology inconsistency, unreliable LLM-judge grading) are exactly the traps to avoid when I evaluate quantized models or multilingual jailbreak susceptibility. Concretely: if I report that a model becomes “more jailbreakable” after quantization or in a given language, this paper is the reference for why I need to control for judge reliability (are refusals graded consistently across languages and across quantized-vs-full-precision outputs, which may differ in fluency and therefore in how an LLM judge scores them?) and dataset adequacy (is my harmful-prompt set large and representative enough per language/per quantization level to support the comparison I want to make?). It’s a citation for methodological pressure and framing, not a substitute for doing that evaluation work rigorously myself.