Field Notes

Working notes on AI systems, safety research, labor, ethics, and governance policy

Frontier AI Governance Ecosystem Map

Frontier AI is governed through a chain of corporate controls, evaluators, standards bodies, regulators, national-security authorities, and international coordination mechanisms, but the weakest connection remains the step from detecting a dangerous capability to compelling action.

AI Governance BlueDot Impact growing

How Do We Measure an AI Agent That Can Reason About the Test?

METR’s time-horizon work is evolving into a broader measurement program for autonomous agents: task length, economic returns, evaluation integrity, and whether AI can accelerate AI R&D.

AI Safety AI Governance Agents & Tooling BlueDot Impact seeding

Critical Weight Protection for Fairness and Safety under Quantization

ACL 2026 Findings: static/dynamic PTQ degrades fairness and MultiJail safety (worse in KO/AR); Critical Weight Protection keeps top FAIRSCORE+SAFESCORE weights in FP16.

AI Safety Quantization AI Governance growing

Q-resafe: Safety Risks and Patching for Quantized LLMs

ICML 2025: systematic English ASR eval across AWQ/AQLM/LLM-QAT/QLoRA at INT4/INT8; Q-resafe patches safety-critical weights to restore pre-quant refusal.

AI Safety Quantization Multilingual growing

Refusal Is Mediated by a Single Direction

NeurIPS 2024: across 13 chat models up to 72B, a single residual-stream direction mediates refusal; ablating it jailbreaks, adding it forces refusal.

Jailbreak Mechanistic Interpretability Refusal Red Teaming growing

Safety-Critical Parameters (ESI / SET / SPA)

ACL 2026 Findings: ESI ranks safety-critical weights (middle V/MLPs in dense models; late MLP experts in MoE); SET updates ~1% for safety; SPA freezes them during task FT.

Fine-Tuning Mixture of Experts Alignment AI Safety growing

Safety Layers in Aligned LLMs

ICLR 2025: contiguous middle ‘safety layers’ distinguish malicious vs benign queries; SPPFT freezes them during fine-tuning to limit security degradation.

Fine-Tuning Mechanistic Interpretability Alignment AI Safety growing

GPT-Red and the Scaling of Automated Red-Teaming

Technical field note on OpenAI GPT-Red (Jul 2026): self-play RL attacker for direct/indirect prompt injection, Fake CoT, adversarial training of GPT-5.6 Sol; implications for red-teaming research programs and role structure.

Prompt Injection Reinforcement Learning Red Teaming Alignment growing

Research Log Terms: A Plain-Language Primer

Plain-English glossary for the technical terms in Lit Review 1.0 and 1.1: quantization, GGUF, refusal directions, middle layers, Δ_HL, and related measurement terms.

AI Safety Quantization Multilingual Surveys growing

Research Log: Redteaming Quantized Models with Multilingual Inputs

Status report: measuring Δ_HL (low- vs high-resource jailbreak ASR) under GGUF and related quantization arms.

Red Teaming Quantization Multilingual Agents & Tooling growing

Literature Review of Core Quantization and Multilingual Safety Anchors

25 published papers across 17 venues (2023-2026), merging Lit Review 1.0 and 1.1.

Red Teaming Quantization Multilingual Surveys growing

Setting up my Digital Brain Karpathy Wiki with RAG system and local DB, running on local models

How I turned my CarbonInterface learning journal into a locally-run RAG system

RAG Agents & Tooling growing

Literature Review 1.1: Quantization Baselines, Agentic Red-Teaming, and the Labor-Attribution Crack

Al Hakim (ACL 2026 Findings) runs MultiJail EN/KO/AR under PTQ and partially fills quant×multilingual safety; Q-resafe remains English ASR. Full 9-lang Δ_HL × GGUF still the sharper open claim.

Red Teaming Quantization AI Governance Surveys growing

Single-Layer RL Can Match Full-Parameter Training

arXiv:2607.01232 (Is One Layer Enough?); RLVR gains concentrate in middle transformer layers; training one layer with GRPO/GiGPO/Dr. GRPO often matches or beats full-parameter RL on math, code, and agents.

GRPO RLVR Reinforcement Learning Alignment growing

STLA: Spatiotemporal Lookahead Alignment for Post-Training Quantization

ICML 2026: rounding-optimized PTQ that collocates learning and compensation via Hessian-guided clustering and a Schur-Complement lookahead objective.

Post-Training Quantization Quantization growing

Detecting the Semantic Fixed Point

ICML 2026 Oral: training-free early exit from the geometry of hidden-state trajectories (update norm + cosine alignment), not output confidence.

Early Exit Quantization growing

Reranker Helps, but Not Enough

ICML 2026: P³A poisoning attack that defeats benign-trained rerankers via rule-based prompts plus ~1% character-level perturbations, transferable to vanilla RAG.

Data Poisoning RAG Red Teaming Agents & Tooling growing

Evaluating the Safety of LLMs in Healthcare and Dentistry

BDJ Open 2026: narrative review of prompt-based red-teaming for clinical LLMs, with a red-blue-purple lifecycle framework for dentistry and healthcare.

Red Teaming AI Safety growing

Compliance without Coherence

AI and Ethics 2026: fluent compliance can mask incoherent internal reasoning; the monitoring layer of alignment evaluation has a principled blind spot.

Alignment AI Safety AI Governance growing

SoK: A Taxonomy of LLM Threats

Computer Science Review 2026: lifecycle-centric SoK of 20 LLM attack classes across training, inference, and system integration, with a dependency-aware defense-in-depth agenda.

Red Teaming Surveys growing

LogiCP: Formal Logic Inference Guided UQ for Personalized Federated Learning

JAIR 2026: STL-based semantic client clustering plus decentralized conformal prediction for personalized FL, evaluated on traffic, temperature, and electricity forecasts.

Federated Learning Formal Methods Uncertainty Quantification AI Safety growing

My Google Interview: A Multi-Agent System for Staying Ahead of AI

A design for a multi-agent pipeline that sources, vets, and scaffolds AI curriculum as fast as the field actually moves, plus metrics for measuring behavior change instead of course completions, plus a filterable table of the actual sources worth watching.

Agents & Tooling growing

Matt Pocock's Skills, Actually Explained: A Critical Guide for Current Usage

A dig through Matt Pocock’s mattpocock/skills repo (the real SKILL.md files, not the SEO blogspam about it): what the skills actually do, what the community has genuinely pushed back on, and what’s worth stealing.

Agents & Tooling growing

Literature Review 1.0: Red-Teaming, Quantization, and Multilingual Robustness

Phase 1 - Research Niche Selection & Literature Foundation.

Red Teaming Quantization Multilingual Surveys growing

When Search Goes Wrong

ICML 2026 — CREST-Search red-teams web-augmented LLMs by manipulating search queries to induce unsafe citations, not unsafe generation.

Red Teaming growing

Gray-Box VLM Adversarial Alignment

ICML 2026 — SVD-structured adversarial attacks against gray-box vision-language models.

Red Teaming Alignment growing

LLM Safety Evaluations Lack Robustness

ICML 2026 Position paper — the current LLM safety-evaluation pipeline lacks robustness across dataset curation, red-teaming, generation, and judging.

Alignment growing

Chain-of-Thought as a Lens

ACL 2026 — a semantic-entropy-based ‘Alignment Score’ for measuring how well multi-step LLM reasoning tracks human-preferred reasoning, and where it breaks down.

Chain-of-Thought Alignment growing

Bridging the Scale Gap

FAccT 2026 — hybrid human+automated red-teaming to surface latent risks in text-to-image models that neither approach catches alone.

Red Teaming growing

The Last Labour Frontier

AI & Society — generative AI unevenly pushes occupational groups toward the periphery of the labor market.

AI Governance growing

Code-Switching Red-Teaming (CSRT)

ACL 2025 — mixing up to 10 languages inside a single red-teaming prompt beats monolingual attacks by 46.7%, and exposes a resource/safety correlation.

Red Teaming Multilingual growing

Mind the Gap: A Practical Attack on GGUF Quantization

ICML 2025 — extends the quantization-exploitation attack to GGUF, the format actually used by ollama and llama.cpp in the real world.

Red Teaming Alignment Quantization growing

How 'Hard' Are Hard Laws?

Computer Law & Security Review — comparative legal analysis of hard-law vs. soft-law AI governance in South Korea and Japan.

AI Governance growing

Opening the Scope of Openness in AI

FAccT 2025 — a 98-concept taxonomy showing AI’s ‘openness’ discourse leans on open-source software framing that doesn’t actually transfer, and quietly underweights fairness and diversity.

AI Governance growing

100 Days of A.I. Engineering

New reading list and code snippets for building LLMs in production

Agents & Tooling seeding

Exploiting LLM Quantization

NeurIPS 2024 — a benign full-precision model can be crafted so that it turns malicious only after a user quantizes it.

Red Teaming Alignment Quantization growing

How Does Quantization Affect Multilingual LLMs?

EMNLP 2024 Findings — quantization harms are disparately distributed across languages, worse than automatic metrics show, and human evaluators catch what benchmarks miss.

Alignment Quantization Multilingual blooming

Multilingual Jailbreak Challenges in Large Language Models

ICLR 2024 — introduces MultiJail, the first multilingual jailbreak dataset, and shows low-resource languages are roughly 3x as likely to elicit unsafe output.

Red Teaming Alignment Multilingual growing

Jailbroken: How Does LLM Safety Training Fail?

NeurIPS 2023 (Oral) — the foundational taxonomy of why safety-trained LLMs still jailbreak: competing objectives and mismatched generalization.

Red Teaming Alignment growing