Setting up my Digital Brain Karpathy Wiki with RAG system and local DB, running on local models

I’ve kept a “CarbonInterface” technical journal on GitHub every year since 2015 with logs of whatever I was learning, reading, building, and exploring. This also includes years of debugging at various tech jobs, threads of copied over and crossed out approaches from Stack Overflow, and iterations of refactoring to better understand how to make code more clean.

This year, I started implementing Andrej Karpathy’s llm-wiki method for summarizing my earlier notes with an LLM. However, I found this solution to be inadequately detailed when it came to breaking down technical notes. I started creating a RAG database for my notes so that I could chat with it, and it could both point to summaries and specific code blocks. I recently watched Nate Jones’ Open Brain (OB1) episode on YouTube, which takes a similar approach with generating embeddings and storing them in a database for vector search. I’m testing his approach, and making a few improvements, which I’ve contributed as pull requests on his repo.

One major difference from Nathan Jones’ Open Brain is that I prefer to have everything run locally, for privacy. Embeddings and chat both go through Ollama on my machine (mxbai-embed-large for vectors, qwen for answers), so the interface does not depend on Claude or any other cloud chat client.

Here’s the way I’ve architected my project:

Brain Architecture

In this approach, compared to Karpathy’s, the database is the actual source of truth, and the wiki is a build artifact. Karpathy’s LLM-wiki idea also treats the wiki as an emergent, regenerable view over atomic state, not a thing you edit directly. However, I keep breaking this rule, hand-editing the wiki pages for better wording, because I find the LLM-summaries are still quite inadequate and miss key points.

Going fully local

Nathan’s OB1 assumes you’re comfortable sending your notes out to OpenAI or OpenRouter for embeddings, billed per call. I wanted this local, because eleven years of the dumb ways in which I’ve been stuck on various technical problems isn’t exactly something I want living on someone else’s inference endpoint by default. So:

• 1024-dim vectors via local Ollama (mxbai-embed-large) instead of 1536-dim OpenAI embeddings. Not interchangeable: switching embedding models later means re-embedding the entire table, no partial migration.
• The MCP server runs as a local stdio process, not a cloud edge function, because an edge function has no way to reach a model running on my own machine.
• Entity extraction and edge classification run as local scripts: the upstream versions of those workers call out to cloud LLMs by default.
• A nightly job checks whether Ollama is actually running and starts it if not. Past me forgot to leave it running more nights than I’d like to admit, and woke up to a failed log instead of an updated wiki.

Scheduling automations

A scheduled job runs every night: import whatever changed in the vault, drain the entity queue, classify new reasoning edges, regenerate the entity and topic wiki pages. It’s persistent, so if my laptop was asleep at compile time, it just catches up next boot.

In the past, I’ve used Claude CoWork as well as OpenClaw and Hermes for scheduling some routine morning tasks, to generate a daily brief that reads my calendar and email, generate a to do list and schedule for my day, and help me prioritize my most urgent tasks. I am now testing OpenWork for this purpose, again, to minimize on token costs, because I don’t want to build a dependency habit when costs inevitably go up.

I use these automations to pull from my Google Scholar alerts and help me begin to triage which recent research papers are worth reading, which ones come from Tier 1 journals or are preprints on Arxiv, which ones touch on ideas that I’m currently exploring right now. This is a helpful starting point, but often citations are recorded incorrectly, datasets and methodologies are hallucinated, and overall, I treat this as an untrustworthy research assistant/intern, but not as a reliable source of judgment on my research. It saves me time to have these Google Scholar summaries get generated every day and stored in my improved Carbon Interface, but the quality of these notations depends on how much time I am able to devote on weekends to evaluating and improvement the outputs.

This is part of the same Scholar-alert pipeline from my Week 1 literature review post, just further downstream: alerts land in Gmail, the digest reads them, and the ones worth keeping get pulled into the vault as their own atomic thoughts instead of dying in an inbox.

There’s a Learning Biography the compiler regenerates too, a standing synthesis of just the technical material (coursework, the AI-news archive, book reviews, the reading log), with personal journal content deliberately excluded. Overall, this is a helpful way to organize the various research ideas I am examining each week, and collect the papers I’m reading in one place, with subject and methodology labels.

Contributing back to Nathan Jones’ open source repo

I’ve made some small usability edits to Nathan’s Open Brain project. Three PRs I’ve opened on his repo:

• #443: the dashboard’s kanban board had a real bug. Dropping a card onto another card (instead of into empty column space) silently failed, which meant a full column was effectively unreachable. Fixed the drag handler, finished the intra-column reordering that was scaffolded but never wired up, and fixed a genuinely funny bug where the card-edit modal rendered nearly transparent over the board.
• #444: a reading-list schema that had been sitting in the “planned” column of the schemas README. Books, articles, podcasts through want-to-read → reading → finished, plus abandoned, because DNF-ing a book is a real outcome and I didn’t want a future ALTER TABLE just to admit that.
• #445: finished the daily-digest recipe’s Supabase Edge Function path, stubbed as “planned, contributions welcome” since the recipe was written. Kept it template-based rather than routing it through an LLM call on purpose, a free, zero-marginal-cost path for anyone who wants a scheduled digest without needing an API key for anything.

Why bother

Eleven years of technical learning logs are only worth as much as they’re searchable. Now when I’m stuck on something, I can just ask my Carbon Interface, now enabled with chat, and it can tell me whether past-me already worked on something similar, and what approaches led to dead ends, or have yet to be explored.