Week 1 Literature Review: Red-Teaming, Quantization, and Multilingual Robustness

Classification

Most relevant to my research area

Categories A, B, C, and D (6 papers) are the core of my area and the ones I’d lead with in the meeting. Everything in E–H is real, verified, Tier-1 work, but adjacent — useful for methodology or framing, not a direct empirical anchor.

Ideas worth discussing

• The theory already predicts the gap should compound, but nobody has tested it. Jailbroken’s “mismatched generalization” argument (D) is that capability outruns safety training wherever safety-training data is thinnest — which is explicitly the low-resource-language case (A). Quantization (B) independently degrades things unevenly by language, worse than automatic metrics show (C). Nobody has tested whether these two independent, unevenly-distributed-by-language effects compound when stacked.
• Code-switching (A) is a naturalistic attack, not an adversarially engineered one — CSRT’s framing that some of the most effective “attacks” are just underrepresented natural input patterns is a genuinely interesting reframe of what “adversarial” means in a multilingual context, and it’s untested against quantized models.
• GGUF is what’s actually deployed (B, via Mind the Gap) — the quantization-as-attack-surface literature already targets the real-world local-LLM deployment path (ollama/llama.cpp), which makes a multilingual extension immediately practically relevant, not just theoretically interesting.
• Evaluation rigor (F) is a prerequisite, not a footnote. The ICML 2026 position paper’s four failure points (dataset curation, red-teaming optimization, generation, LLM-judge grading) all get worse, not better, when you add languages and quantization levels as additional variables — judge reliability across languages and across full-precision-vs-quantized fluency differences is a real confound I’d need to control for, not assume away.

Candidate directions for a novel, Tier-1-worthy project

1. Primary candidate — Quantization × multilingual attack-success-rate, tested jointly. This is the gap the search kept surfacing and never filled: MultiJail/CSRT test multilingual jailbreak susceptibility without quantization; the EMNLP 2024 Findings paper tests quantization × multilingual without jailbreak/attack-success (only quality degradation); Egashira et al. test quantization-as-attack without a language dimension. A study that quantizes multilingual open-weight models across schemes/bit-widths (int8, int4, GGUF) and measures jailbreak attack-success-rate — not just quality — across a resource-tiered language set (extending MultiJail’s 9 languages, or CSRT’s code-switching design) would be the first paper to jointly test this. Testable hypothesis, derived directly from Jailbroken’s theory: quantization widens the existing ~3x high-resource/low-resource safety gap, because compression and thin safety-training coverage compound rather than act independently. Needs rigor-bar controls from category F (per-language, per-precision judge reliability) baked into the design from the start, not bolted on after.

2. More novel, higher-risk/higher-reward — a “linguistically-triggered” quantization attack. Egashira et al.’s constraint-projection technique (B) builds a full-precision checkpoint that’s benign until quantized. Nobody has combined that with language as the trigger variable rather than quantization alone. Candidate project: can an attacker construct a model that is safe under evaluation in high-resource languages at full precision and post-quantization, but reliably unsafe specifically when queried in a targeted low-resource language after quantization — i.e., the trigger is the intersection of compression and language, not either alone? This would be a genuinely new threat model, not an incremental benchmark extension, and it directly combines two already-verified Tier-1 techniques (A + B) rather than starting from scratch.

3. Lower-risk, still publishable — a rigor-first evaluation benchmark paper. Apply category F’s four-stage critique explicitly to the quantization × multilingual space: build (or extend) a dataset and evaluation protocol specifically designed to avoid the failure modes ICML 2026’s position paper identifies, with per-language and per-precision judge-reliability checks as a first-class design constraint rather than an afterthought. This is the safest option to actually finish and publish, and could serve as the evaluation infrastructure either of the above two directions would need anyway.

Proposal to lead with #1 as the research paper’s primary empirical claim, keep #2 as the “if the results are interesting, here’s the follow-up” pitch, and treat #3 as something I might need to build regardless, as infrastructure.

Research Log

Week of 06/29/26 to 07/03/26

1) Major accomplishments for the week

• Completed Unit 1, parts of Units 3 and 4, and began the hands-on sections in Unit 5 (Synthetic Datasets)
• Set up Google Scholar with alerts for my areas of research.
• Posted notes for 13 papers found using Google Scholar to blog, with a Week 1 summary.

2) Hours spent on research during the week

10

3) Any problems encountered and how did you overcome them?

Creating better system for filtering papers from top research journals, and evaluating interesting notes. Need to gather citations for each paper, with a system for verification.

4) Required goals for the coming week

Decide on research direction with Dr. Bhuva, and understand the most important foundational knowledge for implementing research in my area.

5) Stretch Goals for the coming week

Begin creating synthetic dataset for multilingual inputs.

6) Question for Dr. Bhuva

Is this a framing that you think is worthy for a Tier-1 submission? Will this area for research close before I am able to generate the results? Do I need more infrastructure/equipment for quantized LLM testing than I can afford? Should I keep the quantization angle or focus only on multilingual inputs?

Please evaluate secondary paper topic for implementing / evaluating Zoe Cullen’s policy paper from a technical standpoint. Is this a paper I can also work on, after this initial paper on multilingual LLM redteaming? I am passionate about this topic because of its implications for labor policy, which is relevant to my field, and the possibility of reaching out to Dr. Cullen.