Jessica Quaye, Alicia Parrish, Charvi Rastogi, Minsuk Kahng, Oana Inel, Lora Aroyo, Vijay Janapa Reddi — Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT 2026), DOI: 10.1145/3805689.3812210. Venue and full author list confirmed via the FAccT 2026 proceedings listing and a co-author’s (Minsuk Kahng’s) publications page. My earlier note flagged the venue as unconfirmed (“ACM Conference, 2026 — likely FAccT or CHI”); it is now confirmed as FAccT 2026, not CHI.

Core Contribution

The paper’s framing rebuts “just pick one” debates about red-teaming strategy: pure human red-teaming has judgment and nuance but doesn’t scale to the volume needed to characterize a deployed system’s risk surface; pure automated red-teaming scales but misses the latent, context-dependent harms that require human interpretation to even recognize as harms. The contribution is a hybrid, human-in-the-loop methodology that uses automated techniques to augment human red-teamers — extending their reach rather than replacing their judgment — applied to text-to-image (T2I) generative systems.

Method

Based on the confirmed title and framing, the approach sits in the “human-in-the-loop” category of red-teaming methods, distinct from purely manual approaches (e.g., Adversarial Nibbler) or purely automated distribution-modeling approaches (e.g., DREAM). The design logic — evident from the title itself — is to use automation to scale red-teaming coverage (more prompts, more variation, faster iteration) while keeping humans in the loop for the judgment calls about what counts as a “latent risk,” which is exactly what automated classifiers tend to miss. I was not able to retrieve the full published methods section (the FAccT proceedings listing and co-author publication page gave only citation and DOI, not full text), so I’m flagging as unconfirmed the specific automation technique used, the T2I models evaluated, and any quantitative results — these require a direct read of the proceedings PDF.

Limitations

  • I could not verify methodological specifics (which T2I models, how “latent risk” is operationalized as a measurable outcome, sample sizes). This entry is confirmed on bibliographic facts (real paper, real venue, real authors, real DOI) but not yet confirmed on methodological depth.
  • Domain mismatch with my core text-only LLM focus: this is T2I, not language-model red-teaming. What transfers is the hybrid methodology, not the specific failure modes found in image generation.
  • Human-in-the-loop methods carry a structural cost/scale tradeoff by design — the paper’s own premise is that this is a deliberate compromise, not a free lunch. It likely doesn’t scale as far as fully automated red-teaming even where it catches more.

Relevance to My Niche

Moderate, methodological relevance rather than direct subject-matter relevance. My niche is red-teaming LLMs across languages and quantization levels — a different modality than T2I — but the underlying design question is shared: who or what is best positioned to recognize a given failure mode, and does that change with scale? For multilingual red-teaming specifically, an analogous hybrid design looks necessary: automated translation/back-translation can scale coverage across many languages, but recognizing whether a translated jailbreak actually landed as intended (versus mistranslating into something harmless, or something differently harmful) plausibly needs a human fluent in that language in the loop — exactly the human/automation division of labor this paper is structured around. I’d treat this as a landscape/methodology reference rather than a direct empirical anchor for the niche.