My Google Interview, or Curriculum at Model Speed A Multi-Agent System for Staying Ahead of AI

Context: My Google interview this week

I had an interview this week with Google’s GET team (their internal technical enablement group for Google Cloud) and one challenge the interviewer, Angela Karam, flagged stuck with me: keeping pace with AI developments while still producing quality enablement is genuinely one of the harder open problems in the field right now. Traditional curriculum design is intentional and slow by design, but the field changes weekly. Formal class participation is dropping across the industry because nobody wants to sit through a three-hour course that might be obsolete by the time they finish it. And underneath all of that sits an even harder problem: measuring whether any of it actually changed how people work, rather than just whether they clicked through to the end.

That’s not a Google-specific problem. Every team teaching AI skills right now, mine included, is fighting the same fight: the shelf life of a slide deck about “how to prompt an agent” is measured in weeks, not years. So I wanted to actually design the thing I’d build if someone handed me the budget and the mandate: a system for sourcing knowledge fast enough to matter, turning it into curriculum without a six-month lag, and proving it changed behavior instead of just proving people showed up.

This blog post explores those challenges and proposes some solutions that I use in my personal learning management stack. Use the contents box below to jump around. There are three parts: the multi-agent pipeline itself, ideas for measuring enablement in the actual flow of work instead of a course, and, since this is the part I’d actually pay someone to get right, a full filterable table of sources worth watching and how to get a few months of lead time on what’s coming.

The Curriculum Decay Problem: why this is different from “keep the docs updated”

Most content-maintenance problems are decay problems: the software changed, the doc didn’t, fix the doc. AI curriculum has that problem too, but it also has a much weirder one: the thing you’re teaching keeps getting superseded by a fundamentally different way of doing the same task. You’re not just updating “here’s the current API,” you’re deciding whether last quarter’s whole approach (say, manual prompt chaining) is still worth teaching at all now that agentic tool-calling does most of it better. That’s an editorial judgment call, over and over, at a pace no curriculum committee can keep up with by meeting monthly.

The other wrinkle is audience spread. In any org past a few hundred engineers, you’ve got people who’ve never touched an AI tool next to people running agentic pipelines in production, and one-size-fits-all training actively hurts the second group. This is the expertise reversal effect from cognitive load theory: instructional supports that help novices measurably slow down experts. So whatever you build has to source content fast, judge what’s actually durable versus what’s hype-of-the-month, and then fork the delivery by where the learner already is. That’s the design problem underneath everything below.

Part 1: Multi-agent pipeline for sourcing, vetting, and scaffolding curriculum

The pipeline has eight stages. Steps 1–8 below follow the same order as the diagram:

1. Sourcing: five scout agents, one per source type
2. Credibility + dedup: tier scoring and story merging
3. Synthesis: per-item summaries and the weekly digest
4. Trend-forecasting: what matters in 8-12 weeks
5. Curriculum scaffolding: competency mapping and durable vs. frontier calls
6. Lab generation: cloud labs built like software
7. Editorial review: human-gated pedagogical + technical passes
8. Decay / deprecation: keep the curriculum regenerable

Here’s the pipeline, roughly in the order information flows through it:

Step 1. Sourcing layer: five scout agents, one per source type

Each scout runs on its own schedule and its own logic, because “check for new stuff” means something completely different for arXiv than it does for X. I’d build these as genuinely separate agents rather than one agent with five tools, because each one needs different judgment baked in, not just a different API call.

1.1 Scholar / arXiv Scout

Runs against a set of standing Google Scholar alerts and the arXiv API directly (cs.CL, cs.AI, cs.LG, plus whatever subfields are actively relevant, like agents or evals this quarter). Doesn’t just grab title/abstract, it pulls citation velocity from Semantic Scholar (how fast is this being cited relative to papers from the same week) as an early credibility signal, because a paper nobody’s citing three weeks in is a much weaker candidate than one already getting picked up by other labs.

1.2 YouTube / Podcast Scout

This is the one I have the most opinions about because I’ve basically already half-built it for my own use. Pull new uploads from a curated channel list (not a search, a list, curation happens once at setup and gets reviewed quarterly), run them through local Whisper for transcription (I already run Ollama locally for embeddings on my own note-taking setup, so bolting a local ASR step onto the same box is a natural extension), then hand the transcript to the synthesis agent rather than trying to summarize video directly. The reason to transcribe rather than just watch: a transcript is searchable, quotable, and diffable against future videos on the same topic, a video isn’t.

1.3 X / Bluesky Scout

The trap here is firehose: following a big list and drowning. Instead, this scout watches a small, deliberately curated list of individual researchers and lab accounts on X or Bluesky (never aggregator accounts, they’re already downstream of the primary sources), and specifically weights replies and quote-posts from other credible accounts higher than the original post’s own engagement numbers, since researcher pushback or confirmation is a better signal than like counts.

1.4 Newsletter / Blog Scout

Subscribes to sources in the table below via a dedicated inbox, parses new issues, and extracts links rather than trusting the newsletter’s own summary, since a lot of newsletter content is itself downstream synthesis of the same primary sources the other scouts already caught, that overlap is exactly what the dedup agent is for.

1.5 GitHub / HN / Trending Scout

Watches GitHub trending (filtered to AI/ML topics), the Hacker News front page, and r/MachineLearning and r/LocalLLaMA, not for news, but as a social proof-of-relevance signal: something the other scouts flagged as maybe-important that’s also trending here is a much stronger candidate for “teach this now” than something only one source mentioned.

Step 2. Credibility and dedup: the least glamorous, most important agent

Every item arriving from the scouts gets scored against a source-tier rubric before anything else happens to it, strongest signal at the top:

1. Primary research (peer-reviewed, or a preprint from a named lab with reproducible results)
2. Lab/vendor primary sources (official release notes, model cards, official blog posts, this is ground truth for capability claims, but treat competitive framing skeptically)
3. Named practitioner synthesis (a specific person with a track record writing analysis, not just aggregation)
4. Aggregator/newsletter digest (useful for triage, not for citation, always trace back to what they’re citing)
5. Social/community chatter (a leading indicator of what’s resonating, never a source of fact by itself)

Dedup then merges the same underlying story across tiers (a model release announced at tier 2, analyzed at tier 3, and trending at tier 5 is one story, not three), and flags outright contradictions between sources for human review rather than silently picking a winner.

Step 3. Synthesis and the weekly digest

Each surviving item gets a structured per-item summary (what changed, why it matters, who it affects, confidence level), and once a week those roll up into a single digest, organized the same way my own research log already is: by category, not by source, so a reader sees “AI Engineering,” “AI Safety,” “Local Models,” not “arXiv this week” followed by “Twitter this week.”

Step 4. Trend-forecasting: the agent trying to see two months out

This is the agent doing the job the “most important” part of this post asks about directly, so I’ve given it its own section further down (see Predicting a few months out). Structurally, it’s a separate agent from synthesis because forecasting needs a different objective: instead of “summarize what happened,” it’s scoring “how fast is the rate of mentions/citations/commits changing,” which is a derivative, not a snapshot.

Step 5. Curriculum scaffolding: this is where it becomes teaching, not news

Once a topic clears the credibility bar and the forecasting agent flags it as worth teaching now (not just interesting now), the scaffolding agent does the actual instructional design work:

• Maps the topic onto the org’s existing competency model (I’d use something like: tool operation, critical evaluation, workflow integration, building AI-native features, each with 3 to 4 proficiency levels) rather than inventing a new objective from scratch every time.
• Writes the learning objective as an observable behavior, not a topic, using backward design: start from the outcome, then define the evidence for it, then design the activity, in that order. “Engineer can evaluate whether a RAG pipeline’s retrieval step is the actual failure point versus the generation step” is designable and testable this way. “Engineer understands RAG” is neither.
• Makes an explicit call on durable versus frontier: does this belong in the stable curriculum library, or does it belong in a fast, disposable format (a one-page internal post, a short recorded walkthrough) because the tool itself will look different in six weeks? Forcing this content into a heavyweight course format is how curriculum teams fall behind, some things genuinely deserve a one-pager and nothing more.
• Forks delivery by prior fluency where the topic warrants it, so the same underlying content doesn’t punish either a novice or an expert with the wrong scaffolding level.

Step 6. Lab generation: cloud labs, built like software

For anything that clears the bar for hands-on practice, this agent drafts a lab spec against a fixed discipline rather than freehand:

• States the objective as an observable end-state (“learner can configure a retrieval eval and identify which of three failure modes is occurring”) before anything else gets written.
• Classifies every step of the intended lab time as target skill, prerequisite skill, or pure environmental noise, and pushes hard to get target time above 60% of the lab (first drafts of any lab tend to land under 30% without this discipline).
• Specifies state-based validation (check that the eval actually caught the injected failure, not that the learner typed a specific command), because command-matching validation quietly teaches that there’s one true incantation, which is the opposite of the judgment you’re trying to build.
• Generates the environment spec as pinned, ephemeral, and disposable: a fresh sandboxed environment per learner per attempt, torn down after, with every tool and model version pinned, since unpinned AI labs rot faster than almost any other kind of lab.
• Writes a CI harness that runs the lab’s own steps and validation on a schedule, so breakage gets caught by a scheduled job instead of by the thousandth confused learner.

Step 7. Editorial review: two axes, run in parallel, never merged

Before anything ships, it goes through a human-gated review with two separate passes that don’t get reranked against each other: a pedagogical soundness pass (is the objective observable, is the scaffolding right for the intended audience, does the lab’s target time clear the bar) and a technical accuracy pass (does this actually reflect current model behavior, is the claim reproducible, would a subject matter expert sign off). Keeping these separate matters because content can pass one and fail the other, technically flawless content with a bad learning objective is just as broken as a beautifully scaffolded lab teaching something the model doesn’t actually do anymore.

Step 8. Decay and deprecation: treat the curriculum like a wiki, not a library

This is the piece I’d take most directly from my own note-taking setup: the curriculum itself should be a regenerable view over a source-of-truth knowledge base, not a pile of hand-maintained slide decks. A dedicated agent watches every shipped unit for staleness signals (model version drift, falling completion or pass rates, a scout flagging that the underlying tool behavior changed) and either flags it for the scaffolding agent to regenerate, or archives it outright. A reasonable planning assumption is that AI-tooling content costs something like 20 to 30 percent of its original build cost per year just to stay current, and for the fastest-moving topics it runs higher than that, which argues for fewer, better-instrumented units over a sprawling library nobody’s maintaining. Curation has to include deletion, or the library becomes a graveyard with a search box.

Part 2: Measuring enablement in the flow of the job, not through a course

This is the harder problem, and it’s the one most programs quietly give up on, because it’s much easier to report “4,000 completions” than to prove anyone works differently now. The Kirkpatrick Model is the standard vocabulary for this, and it’s worth having on hand precisely because it shows where most programs stop:

Almost every program instruments Level 1 and 2 well (smile sheets, quiz scores) and stalls out before Level 3. The thesis I’d build around: AI-assisted engineering leaves a telemetry trail that almost no other training topic does. Every AI-tool interaction, every accepted or rejected suggestion, every agentic workflow invoked, is an event already sitting in the developer toolchain. The hard part isn’t collecting the data, it’s defining what “fluent” even means before you start counting anything.

A few concrete ideas for measuring this in the actual flow of work:

2.1 Measurement approaches

1. Modification rate on accepted AI suggestions, not just acceptance rate. A fluent engineer edits AI output; blind acceptance and blanket rejection are both anti-signals, not opposite ends of a good spectrum.
2. Revert and defect rates on AI-assisted changes, benchmarked against a baseline, ideally paired with a staggered rollout across teams so the rollout schedule itself becomes the identification strategy: compare trained versus not-yet-trained cohorts on the same telemetry stream, rather than a before/after on the whole org at once.
3. In-flow micro-assessment instead of a course: a short, scenario-based “here’s AI-generated code with a subtle defect, find it” prompt, delivered inside the tool at a natural pause point, sampled across the population rather than run as a census. This measures the critical-evaluation capability that passive telemetry can’t see at all, and it’s a learning event in its own right (retrieval practice, testing as a way of learning, not just checking), not just a measurement.
4. Support and question-deflection trends. Are the same basic questions showing up less often in the team’s help channel over time? That’s a workflow-integration signal that a course completion certificate will never capture.
5. Peer-teaching signal, not just authored-content signal. If your org has any equivalent of Google’s g2g peer-teaching culture, internal wiki edit frequency and how often people are answering each other’s questions in chat is a real communities-of-practice measure, and it scales in a way authored content never will.
6. Report everything at team or cohort level, never as individual performance data. The moment this telemetry becomes something that shows up in someone’s review, people start gaming it and the signal you built the whole system to get dies immediately, Goodhart’s law in miniature: once a measure becomes a target, it stops being a good measure. This is a trust design constraint, not an ethics footnote you bolt on afterward.
7. Resist a single composite score in year one. A single “AI Fluency Index” hides the fact that tool operation, critical evaluation, workflow integration, and building AI-native features are genuinely different capabilities that can move independently, and a senior engineer’s fluency signature looks nothing like a new grad’s even when both are genuinely fluent. Report the dimensions separately until the model is validated against real outcomes.

None of this replaces the Kirkpatrick levels, it’s an argument for actually reaching Level 3 instead of stalling out at Level 1 the way most programs do. Level 4 claims should stay modest, attribution across a large org is genuinely hard, and overclaiming it is the fastest way a measurement program loses credibility with engineering leadership.

Part 3: The actual sources, and how to get a few months of lead time

This is the part I actually think matters most. Below is the actual database, click a category to narrow it down or use the search box, followed by the reasoning behind each category and the leading indicators worth tracking. One honest caveat up front: newsletters and curated feeds in this space churn constantly (Papers with Code itself was shut down by Meta in mid-2025 and only came back in 2026 as a community revival, which is as good a proof of this point as any), so treat this as a starting shape to verify quarterly, not a permanent bookmark folder. It lives as a real data file in the repo (data/ai_knowledge_sources.yaml), so adding a source later is a one-line edit, not HTML surgery.

3.1 Primary and preprint tracking

Set up Google Scholar alerts per subfield rather than one broad AI alert, a narrow alert on “agentic evaluation” surfaces different (and more useful) things than a broad alert on “artificial intelligence” ever will. Read the arXiv daily listing by title alone, most of the signal is in noticing which subfields are suddenly producing volume. Use Semantic Scholar mainly for citation-velocity as a credibility and forecasting signal, not just search, and Papers with Code for the applied/reproducibility angle Scholar alone won’t give you (worth knowing it was shut down by Meta in mid-2025 and revived as a community project in 2026, a good reminder that even “permanent” infrastructure in this space isn’t). Conference proceedings and workshop programs directly (NeurIPS, ICML, ICLR, ACL Anthology for ACL/EMNLP/NAACL, and the growing set of agent- and eval-specific workshops) matter beyond the papers themselves: workshop CFPs are a genuine leading indicator, a topic that gets its own workshop track is usually six to nine months from mainstream relevance.

3.2 Lab and company primary sources

The official blogs from Anthropic, OpenAI, Google DeepMind and Google Research, Meta AI, Mistral, and Hugging Face are ground truth for capability claims, and worth reading with a specific skepticism: treat the capability claim as real, treat the competitive framing around it as marketing. Worth widening this past the model labs, too: NVIDIA’s technical blog is the best source I’ve found for infrastructure and inference-engineering detail nobody else covers, Microsoft Research and Cohere both publish real research (not just product news), and Stability AI is worth it specifically for open-weight image, video, and audio releases, a modality most of the above ignore.

3.3 Academic labs and institutes

A tier between “primary preprint” and “practitioner blog” that’s easy to miss: university and lab-affiliated publications doing plain-language writeups of academic research, often faster and more skeptical than industry blogs about the same underlying work. BAIR (Berkeley AI Research) is the standard-bearer here, one blog post per paper, written by the authors, no marketing layer. Stanford HAI is more policy-adjacent, useful for the “so what does this mean” framing academic papers rarely include themselves, and its annual AI Index report is a good primary source in its own right. The Gradient sits a notch further out, an independent magazine (founded by Stanford SAIL researchers) that runs long-form essays and interviews rather than paper-by-paper coverage, useful for the occasional piece that steps back and synthesizes a whole subfield instead of one result.

3.4 Applied and practitioner writing

This is the category that decays fastest, so verify these are still active before building a whole pipeline around any single one. Writers doing genuine synthesis rather than aggregation are worth keeping: Import AI (Jack Clark’s weekly roundup with a policy lens), The Batch (DeepLearning.AI’s applied digest), Ahead of AI (Sebastian Raschka’s deep technical summaries), Latent Space (swyx and Alessio, AI-engineer-focused), Interconnects (Nathan Lambert, inside the frontier labs), One Useful Thing (Ethan Mollick, applied and practitioner-framed), and independent technical writers doing deep single-topic writeups like Simon Willison, Chip Huyen, Eugene Yan, Lil’Log (Lilian Weng), and Jay Alammar (the illustrated-explanation approach to model internals). Sebastian Ruder’s NLP News is the same idea from inside an active research career rather than a pure-writing one. Worth deliberately including a skeptic too, not just enthusiasts: Gary Marcus is the most consistent critical counterweight to lab hype, right or wrong on any given post, and a curriculum built only from sources that agree with each other has a blind spot. For sheer daily throughput, TLDR AI and The Neuron are both genuinely useful skim-in-five-minutes digests, dense and engineering-focused versus practical and workflow-focused respectively. The test for whether any of these is worth keeping: does a week of their archive teach you something a roundup of the primary sources wouldn’t have, or are they just restating the lab blog post in friendlier language? Keep the former, drop the latter.

3.5 Applied AI tooling

A distinct category from “labs building models”: companies building the infrastructure layer developers actually integrate against, and their blogs double as some of the best applied-practice writing available because they’re solving real production problems, not showcasing benchmark wins. Weights & Biases publishes genuinely technical tutorials and case studies on the applied engineering side of ML, not just their own product. LangChain’s blog is the closest thing to a running record of how agent-framework patterns are evolving in practice, worth reading even if you don’t use the framework, the patterns generalize. Pinecone’s Learning Center is the best applied resource I’ve found specifically on RAG and vector search, written to teach the concepts, not just sell the product.

3.6 YouTube and podcasts, as a transcribed corpus, not a viewing habit

The move here is to stop treating video as something you watch live and start treating it as a corpus you pipeline: a curated (not searched) channel list, run through local transcription, fed into the same synthesis step as everything else. Worth including: research-explainer channels that cover individual papers in depth (like Yannic Kilcher or Two Minute Papers), long-form interview shows that go deeper than a news hit (like Machine Learning Street Talk, consistently the least hype-driven interview format I’ve found), conference keynote recordings (product releases and research announcements both tend to get their real technical detail in the keynote Q&A, not the marketing cut), and any channel run by someone who actually builds the things they’re explaining rather than just reacting to them, Andrej Karpathy’s channel is the model here. If you’re doing this locally, a small local ASR model plus whatever LLM you’re already running for the rest of this pipeline is genuinely enough, you don’t need a cloud transcription API for this.

3.7 X, Bluesky, and social feeds

Curate a short list of individual researchers and lab accounts on X or Bluesky rather than following an aggregator, an aggregator account is already downstream synthesis, you want the primary voice. Worth deliberately weighting quote-posts and replies from other credible accounts higher than raw engagement on the original post, disagreement or confirmation from another named researcher is a much stronger signal than a like count. Given how much the research community’s social presence has fragmented across platforms over the past couple of years, this is worth re-auditing every few months rather than assuming whichever platform you picked last year still has the conversation.

3.8 O’Reilly, workshops, and early-release books

O’Reilly’s live events catalog is consistently one of the fastest-turnaround formats for applied, hands-on material, often faster than a published book by a year or more, precisely because a workshop can be rebuilt in weeks. Their online learning platform’s early-release (“raw and unedited”) books are worth watching for the same reason: you get the author’s current thinking months before the final edit, which for a fast-moving subfield is often more valuable than the polished final version. Cloud vendor training programs (hands-on labs tied to their own current tooling) are the same idea from a different angle, they have a direct incentive to keep the content current since it’s selling their own platform.

3.9 Community pulse and trend-detection

Hacker News front page, GitHub trending (filtered to AI/ML), and r/MachineLearning or r/LocalLLaMA aren’t sources of fact, they’re a sentiment instrument: something that’s already been flagged as technically credible by tier 1 to 3 sources and is independently trending in these communities is a much stronger “teach this now” candidate than either signal alone.

3.10 Predicting a few months out

The honest answer is you’re watching derivatives, not levels, rate of change matters more than current volume:

• arXiv submission velocity within a narrow subfield, a sudden jump in papers on a specific technique, ahead of any mainstream coverage of it, is usually the earliest available signal.
• Workshop and conference CFP topics, these get decided six to nine months ahead of the workshop itself, so a new workshop track is a genuine advance signal, not a lagging one.
• GitHub star velocity and fork-rate on new tooling repos, the rate of change matters far more than the absolute star count, a repo picking up stars fast in its first two weeks is a stronger signal than one with a large but flat count.
• Job posting keyword trends. Watching which skills start appearing in job postings before they’re common knowledge is a genuinely underused leading indicator, and it’s exactly the kind of labor-market-grounded signal that’s worked well for me building curriculum before: needs analysis grounded in what postings are actually asking for, not just what’s exciting in a demo.
• Citation velocity outpacing publication age, a paper accumulating citations unusually fast relative to how recently it was published is a stronger forward signal than raw citation count, which mostly rewards being old.

None of these predict with certainty, they’re leading indicators, not oracles, but stacking two or three of them (a technique showing arXiv velocity, a workshop CFP, and early job-posting mentions, all in the same quarter) is about as much lead time as this field is going to give anyone, and it’s enough to start scaffolding curriculum before the rest of the industry has caught up, which is exactly the gap this whole system exists to close.

Caveat

An important caveat here: this multi-agent educational material creation system is ALWAYS a first draft for discovering ideas and creating test materials for content, during a time when content changes quickly, and rigorous material must also be produced on tight deadlines. It is ABSOLUTELY not a substitute for carefully reviewing materials, and running human tests across diverse (multiple intelligence) learners for measuring learning, which is another subject I love to think about, and will write about in a different post.