Research Question
I am testing whether making a model smaller for local deployment (quantization, especially GGUF) hurts safety more in some languages than others, and whether that damage shows up in a specific part of the network that carries the “refuse harmful requests” behavior.
In my research log shorthand that becomes: measure \(\Delta_{\mathrm{HL}}\) under GGUF, and check whether the refusal direction in middle layers takes disproportionate quantization error.
Here is a breakdown of some basic terms for my research in plain English:
Compression and deployment
Quantization
A full-precision open-weight chat model stores most numbers as 16-bit floats (often called FP16). That is a lot of disk and RAM. Quantization stores those numbers with fewer bits (8, 4, sometimes less). The model gets smaller and cheaper to run. Some accuracy is lost. The research question is what kind of accuracy: math scores, fluency, or the habit of refusing harmful requests.
Analogy: compressing a photo. The picture still looks like a cat at a glance. Fine whiskers may smear. Safety behavior can be one of those fine details.
Bit-width: INT8, INT4, FP16
- FP16: roughly the “original” precision I start from before local compression.
- INT8: 8-bit integers. Milder compression.
- INT4: 4-bit integers. Harsher compression; usually more damage.
When a log says “INT4/INT8,” I mean those two compression strengths as experimental arms.
PTQ (post-training quantization)
Compress the model after it is already trained. No full retraining from scratch. Most of the methods in my notes (GPTQ, AWQ, and the GGUF pipeline) are in this family. Contrast: training a model from the start at low precision (less common in my plan).
GGUF
A file format and quantization toolkit used heavily by local runners such as llama.cpp and ollama. If someone downloads a “Q4” model for a laptop chatbot, it is often a GGUF file.
Why it matters for me: a lot of published “quantization × safety” work uses research PTQ recipes (GPTQ/AWQ/…). Those are related but not the same object as what people actually ship locally. My plan focuses on GGUF so the threat model matches deployment.
GGUF type names: Q8_0, Q5_K_M, Q4_K_M
Different GGUF recipes. Rough intuition:
| Name | Rough meaning |
|---|---|
| Q8_0 | Near 8-bit; gentler compression |
| Q5_K_M | Mid recipe; “K” / “M” are packing details |
| Q4_K_M | Near 4-bit; harsher; common “small enough for a laptop” choice |
They do not all err the same way. Degradation is non-monotonic across GGUF levels, meaning going from Q8 → Q5 → Q4 might not make safety steadily worse at every step. Sometimes a middle setting can look oddly bad or oddly fine. That pattern showed up in other LLM compression work (including “Stochastic Monkeys” style findings on Qwen); I want to know if multilingual harmful-rate does the same under GGUF.
Critical Weight Protection / AWQ-trust
From Al Hakim et al. (ACL 2026 Findings): find a small set of weights that matter a lot for fairness/safety scores, keep those weights in FP16, quantize the rest. Their mitigation is often described as AWQ plus that protect step (“AWQ-trust” in my notes).
My stretch goal shrinks that idea further: protect mainly the weights tied to the refusal direction / middle safety band, not a generic large protect set.
Deeper note: Q-resafe is the English “measure ASR under quant, then patch” neighbor, not the multilingual MultiJail paper.
Safety behavior and attacks
Alignment / refusal
After pretraining, chat models get extra training so they refuse certain requests (weapons, crime help, etc.). Refusal is that “I can’t help with that” behavior. Alignment here usually means that safety training stack, not “the model shares my values” in a philosophical sense.
Jailbreak
A prompt (or prompt sequence) that bypasses refusal so the model complies with something it was trained to refuse. Classic English jailbreaks are one threat. My lane adds language and compression as possible weak spots.
Red-teaming
Deliberately trying to make the model fail safely: invent attacks, measure how often they work, report the failures. Can be humans, automated attackers, or both. My “synthetic multilingual red-teaming dataset” means: I build (and later may release) multilingual harmful-request style prompts for that measurement, with provenance so the data is auditable.
ASR (attack success rate)
Share of attack attempts that succeed (model gives harmful compliance, under a defined judge). Higher ASR = less safe under that test. Logs sometimes say harmful-rate for the same family of idea: how often the model produces harmful compliance on a fixed harmful-prompt set.
LLM-as-Judge
Instead of only humans grading every answer, another LLM grades “safe vs unsafe” (or similar). Fast and cheap. Risky: judges can be wrong, and they can treat fluent English differently from other languages or from awkwardly quantized text. My Mini-Project 1 used both human eval and LLM-as-Judge for that reason.
Inside the network: layers and directions
Layers / “the stack”
A transformer chat model is a stack of repeated blocks (often dozens). Early layers, middle layers, late layers. Information flows upward; the last layers are closest to the words that get printed.
When papers say “middle layers,” they mean a band partway through that stack, not the embedding at the bottom or the final unembedding.
Attention V and MLP
Inside each block there is usually:
- Attention, including a value (V) pathway that helps decide what information to write forward.
- An MLP (feed-forward network) that further transforms the representation.
Safety Layers and ESI find that a lot of refusal-related structure sits in middle-layer V and MLP weights (with some MoE twists). That is why my logs keep saying “middle V/MLP.”
Residual stream
Think of a running whiteboard the layers keep writing on. Each block reads the board, adds an update, and passes the board up. That shared running state is the residual stream. A lot of mechanistic interpretability talks about directions in that stream.
Refusal direction (Arditi et al., NeurIPS 2024)
Claim in plain terms: across many chat models, “whether we refuse” is largely controlled by one direction in the residual stream. If I erase that direction, refusal collapses (jailbreak-like). If I add it, the model refuses even harmless asks.
Deeper note: Refusal Is Mediated by a Single Direction.
When I write “projection onto the refusal direction,” I mean: measure how strongly a layer’s activations point along that refusal axis. If quantization scramble that axis in middle layers more than elsewhere, I have a layer-resolved story, not only a whole-model ASR number.
Activation probing
Read the model’s internal activations (the numbers on that residual whiteboard) while it runs, instead of only reading the final text. Needed to measure the refusal direction under FP16 vs GGUF.
Is One Layer Enough? / RLVR / GRPO
Is One Layer Enough? (arXiv:2607.01232) is about capability training with reinforcement learning on verifiable rewards (RLVR: math/code checks, not safety). GRPO is one RL algorithm in that family. Their finding: updating a single middle layer can match much of full-model RL gain.
Why it appears in a safety log: it is independent evidence that middle depth is special for post-training effects. It is not a safety paper. It is also arXiv-only in my process, so it stays out of the main survey whitelist until a venue accepts it.
Languages and the main metric
Multilingual safety gap
Safety training data and eval are heavily English-centric. The same model can refuse well in English and fold more easily in other languages. MultiJail is a standard multilingual harmful-instruction benchmark (multiple languages). Al Hakim used a 3-language MultiJail slice (EN/KO/AR) under research PTQ. My plan pushes toward a fuller resource-tier language set under GGUF.
High-resource vs low-resource languages
Roughly: languages with lots of internet text and tooling (English, often Spanish/French/…) vs languages with much less. Safety and quality both tend to be thinner in low-resource settings. My hypothesis interest is whether quantization widens that gap.
\(\Delta_{\mathrm{HL}}\) (harmful-rate delta, high vs low resource)
A difference I care about:
\[ \Delta_{\mathrm{HL}} = (\text{harmful-rate in low-resource languages}) - (\text{harmful-rate in high-resource languages}) \](Exact language lists and averaging rules live in the methodology doc.)
If \(\Delta_{\mathrm{HL}}\) grows as I move to harsher GGUF settings, quantization is not just “a bit worse everywhere”; it is uneven by language tier. The log’s sharper claim is that this is a (language × GGUF-type) interaction, not a vague “quantization hurts multilingual safety.”
Non-monotonic
A plot that does not only go up or only go down. Example: Q8 looks fine, Q5 looks worse, Q4 looks less bad than Q5 on some cell. That is why I want multiple GGUF types, not a single “quantized vs not” bit.
Process and infrastructure words
Tier-1
In my logs, Tier-1 means a venue I would treat as a serious target or a serious cite for the quant × multilingual × safety lane (and the adjacent labor fork). In CS/ML, conferences are the main prestige track; journals matter more for the policy/labor side. Workshops do not clear this bar on their own (see the CSRT rule: cite the later ACL main paper, not the NeurIPS workshop poster).
Conferences I count as Tier-1 for this work (CORE A* unless noted):
| Venue | Why it fits my niche |
|---|---|
| NeurIPS | Quantization-as-attack, mechanistic refusal, broad ML security |
| ICML | Quantization methods, safety eval, English quant×ASR (e.g. Q-resafe) |
| ICLR | Representation / safety-layer localization (e.g. Safety Layers) |
| ACL (main) | Multilingual NLP + safety; CSRT landed here |
| EMNLP (main) | Multilingual + quantization quality (e.g. Marchisio et al.) |
| NAACL | Strong ACL-family NLP venue when the claim is language-first |
| AAAI | Broad AI; acceptable if the framing is general rather than NLP-only |
| IEEE S&P (“Oakland”), USENIX Security, ACM CCS, NDSS | If I pitch the work as ML security / audit rather than NLP |
| FAccT, AIES | Direction 3 labor / attribution / fairness framing (not the empirical security core) |
ACL/EMNLP Findings: peer-reviewed ACL-family tracks. I treat accepted Findings papers as Tier-1 cites for the survey (Al Hakim is ACL 2026 Findings). Prestige is a notch below main-conference oral/poster; I note that rather than pretending they are identical.
Not Tier-1 for the main claim: standalone workshops, unrefereed arXiv, and most “position only” industry blog posts. Those can still inform experiments.
Journals / transactions I treat as top-tier when a journal path makes sense:
| Venue | Typical use for me |
|---|---|
| JMLR, TMLR | Longer ML methods writeups; TMLR is rolling |
| Nature Machine Intelligence | High-visibility AI, very selective |
| IEEE TPAMI | Pattern analysis / ML journal track |
| IEEE TIFS, ACM TOPS | Security / privacy / audit framing |
| Computer Science Review, AI and Ethics | Survey / ethics scaffolding (SoK, fluent-failure), not the empirical core |
For the primary \(\Delta_{\mathrm{HL}}\) × GGUF paper, the realistic Tier-1 targets are NeurIPS / ICML / ICLR (mechanism + quant) or ACL / EMNLP (multilingual safety). “Is this Tier-1-worthy?” means: is the claim novel and solid enough for one of those bars, not only a workshop note.
No-preprints rule (for the main survey table)
Working rule for this literature review: the main survey table prefers venue-accepted work. Fresh arXiv papers can be read and field-noted, but they stay tagged / quarantined from the core whitelist until acceptance. That is why “Is One Layer Enough?” informs the idea space without sitting in the main table.
Provenance
Being able to say where a claim or a dataset row came from (which PDF, which alert, which extraction). When the log says survey breadth outran provenance, I mean I could no longer reliably trace table cells back to sources.
Digital Brain / RAG / entity graph
My local knowledge stack: papers and notes in a vault, embedded and searchable (RAG = retrieve relevant chunks, then answer with an LLM), plus an entity graph (papers, methods, datasets as linked nodes). Goal: one verifiable corpus for survey tables and blog notes. Sketch: Digital Brain.
LambdaLabs
Cloud GPU machines I can rent for runs that do not fit a laptop: extracting refusal directions, evaluating GGUF models across languages, etc. Dipen’s note in the log: use that instead of buying hardware for quantization experiments.
Synthetic dataset
Prompts/responses built by a pipeline (with humans and LLMs in the loop). This is what I’m using for multilingual red-team prompts.
Mini-Project 1 / LLM-as-Judge / layoff Q&A
Course project: a Q&A system around layoff-crisis support content, scored by humans and by an LLM judge. Separate from the quant×multilingual paper, but it practices eval habits I need later. Repo linked from the log: layoff-qa-pipeline.
METHODOLOGY.md
The living research plan for the quantized-multilingual project (baselines, languages, GGUF levels, metrics). Drafting it forced the scoping move: do not redo Al Hakim’s PTQ×MultiJail sweep; do GGUF × \(\Delta_{\mathrm{HL}}\), ideally tied to the refusal-direction mechanism.
Interesting Ideas
- Capability RL gains (One Layer) and refusal structure (Safety Layers, ESI, Arditi) independently point at middle depth.
- If GGUF error is uneven across layers, middle refusal machinery may get hit harder than average.
- That turns “does quantization hurt safety faster than capability?” into a measurable plan: per-layer quant error vs per-layer refusal-direction strength, plus whole-model \(\Delta_{\mathrm{HL}}\).
- Defense stretch goal: keep that thin safety-critical slice at higher precision while quantizing the rest (Critical Weight Protection narrowed), instead of inventing a whole new transformer layer.
The open question I am asking: is that layer-resolved refusal-damage result a main mechanistic claim, or only a supporting experiment under a \(\Delta_{\mathrm{HL}}\) × GGUF headline?
Summary
| Log phrase | Plain meaning |
|---|---|
| Quantization / GGUF | Shrink model numbers for local deploy |
| INT4 / Q4_K_M | Harsher shrink settings |
| PTQ | Shrink after training |
| Refusal / ASR / harmful-rate | Did safety hold on harmful prompts? |
| Middle layers / V / MLP | Where several papers say refusal lives |
| Residual stream / refusal direction | One internal “refuse axis” (Arditi) |
| \(\Delta_{\mathrm{HL}}\) | Low-resource minus high-resource harm gap |
| Critical Weight Protection | Keep important weights in FP16 while quantizing |
| Mechanism-targeted defense | Protect the refusal slice, not a new layer |