Digital Brain Local RAG + Regenerable Wiki

I’ve kept a private technical learning journal I call Carbon Interface every year since 2015. Each year’s Carbon Interface includes logs of whatever I was learning, reading, building, and exploring, plus years of debugging at various tech jobs, threads of Stack Overflow approaches crossed out and rewritten, and rounds of refactoring so I could understand how to make the code cleaner. Finding my old logs across many years has proven to be tricky, and I see that some of the problems I’ve solved in one place are solved again in another, years later. I wanted to build a chat interface for my 11 years of notes, to make it easier to search old notes.

This year I started with Andrej Karpathy’s llm-wiki method for summarizing earlier notes with an LLM. It was too thin when the notes got technical, so I built a RAG database I could chat with, pointing at both summaries and specific code blocks. Then I watched Nate Jones’ Open Brain (OB1) episode on YouTube: notes as a database with vector search underneath. It’s very similar to what I had already been building. I’m testing Nathan’s repo, contributing a few upstream fixes, and running the whole path on my local machine. Full narrative: Setting up my Digital Brain….

One major difference in my own implementation of Nathan’s Open Brain is that I want everything local, for privacy. Embeddings and chat both go through Ollama (mxbai-embed-large for vectors, qwen for answers), so the interface does not depend on Claude or any other cloud chat client.

The design choices that matter:

1. Local-only by default. Nathan’s OB1 is happy to ship notes to OpenAI or OpenRouter for embeddings, billed per call. Mine does not. Eleven years of private company bugs I’ve solved in various jobs over a decade, are not something I want living on someone else’s inference endpoint by default. Embeddings are Ollama mxbai-embed-large (1024-dim; switching models later means re-embedding the whole table). Entity extraction, edge classification, and wiki synthesis run on local qwen3. The MCP server is a local stdio process because a cloud edge function cannot reach a model on my machine. Chat sits on top of that local RAG path.

2. Database is source of truth; wiki is a build artifact. Compared with Karpathy’s setup, the database holds the atomic state and the wiki is a regenerable view: entity and topic pages compiled from thoughts, not a thing you edit as the source of truth. In practice I still break that rule sometimes and hand-edit wiki pages for clearer wording, because the LLM summaries miss key points. The next nightly compile can overwrite those edits, which is the reminder that thoughts (or prompts) are what should change, not the rendered page as SoT.

3. Nightly compile + a cheaper daily brief. Every night: import vault delta → drain entity queue → classify edges → regenerate wiki (persistent across sleep; starts Ollama if it’s down). For morning scheduling I’ve used Claude CoWork, OpenClaw, and Hermes; I’m now testing OpenWork so the brief habit does not lock me into rising token costs. The brief still covers calendar collisions, RSVP gaps, email triage, Scholar-alert skim, and yesterday’s unfinished tasks. Scholar keepers become atomic thoughts instead of dying in Gmail (same pipeline as the Week 1 literature review). I treat those auto-triage notes as an untrustworthy research intern: useful for sorting Tier-1 vs arXiv noise, often wrong on citations and methods.

4. Learning Biography stays separate from autobiography. The compiler regenerates a standing synthesis of technical material only (coursework, AI-news archive, book reviews, reading log). Personal journal content is deliberately excluded so a career-learning log does not quietly absorb private reflection, or the reverse.

5. Fix upstream, don’t only fork. I’ve made small usability edits to Nathan’s Open Brain project and opened three PRs rather than routing around the bugs in a private fork: #443 (kanban drag-and-drop), #444 (reading-list schema, including abandoned), #445 (template-based daily digest, zero LLM cost).

Now I can talk to my Carbon Interface of my former self. When I’m stuck or looking for new ideas for hacker projects, I can chat with my digital brain to see whether past-me already worked on something similar, and which approaches led to dead ends or are still open.

Interface

Dashboard overview (counts, workflow snapshot, capture box, recent activity):

Entity graph extracted from the vault (people, projects, topics, tools, orgs, places):

Kanban workflow for tasks and ideas (New → Planning → Active → Review → Done):

Semantic search across the second brain (example query: AI governance):

Architecture

Nightly job: import vault delta → drain entity queue → classify edges → regenerate wiki. Persistent across sleep; starts Ollama if down.

What works well

• Ask-past-me retrieval: semantic search over years of stuck points beats manual rg across annual repos, and chat can surface the same hits in plain language with links back into the vault.
• Local-only data path: embeddings, extraction, and chat stay on-machine, which fits a decade of learning and struggle notes.
• Daily brief as a real habit change: calendar collisions, RSVP gaps, email triage, Scholar-alert skim, yesterday’s unfinished tasks, before I’ve opened either app, with OpenWork as the cheaper scheduler I’m testing now.
• Scholar alerts as an ingestion channel: same pipeline as the curriculum sketch; keepers become atomic thoughts instead of dying in Gmail, with human review for hallucinated citations and methods.
• Learning Biography as a compiled artifact: technical synthesis without absorbing personal journal content (and vice versa).
• Upstream fixes over silent forks: rather than keeping only a private fork, I opened PRs against Nate Jones’ OB1 repo:
  • #443: Workflow kanban drag-and-drop
  • #444: reading-list schema including abandoned
  • #445: template-based daily-digest edge path (zero LLM cost)

Workflow board after the kanban DnD work in #443:

In progress / still developing

Evaluate closely / be wary of

1. Embedding lock-in. mxbai-embed-large at 1024-d is a corpus-wide commitment. Re-embedding is a batch job, not a config flip. Benchmark recall before any model change.
2. Wiki overwrite hazard. Human edits to compiled pages can vanish on the next nightly compile. Prefer editing atomic thoughts or extraction prompts; if I polish a wiki page by hand, I treat it as temporary.
3. Entity extraction hallucination / over-linking. Local LLMs invent tidy graphs from thin notes. Spot-check high-degree entities; prefer typed edges with evidence pointers.
4. Retrieval false confidence. Top-k semantic hits need to be better optimized. Treat RAG as a pointer, not an answer, even when chat phrases it confidently.
5. Daily brief scope creep. Calendar and mail automation that auto-archives or unsubscribes needs a more narrow allowlist and an undo path. Prefer propose → confirm for destructive actions. This hasn’t been a problem for me yet, but I am constantly worried, and want to make sure I can tighten this as much as possible.
6. Ollama availability. Nightly compile fails silently if the daemon isn’t up; the watchdog helps, but failed nights mean a stale wiki. Watch the logs.
7. Personal / technical corpus bleed. Keep Learning Biography filters strict; contamination is hard to reverse once synthesized pages exist.
8. Local agent trust boundary. Stdio MCP and the chat UI still expose vault content to whatever local model session is answering. Treat prompts and tool results as sensitive.
9. Evaluation debt. No frozen eval set yet for retrieval (precision@k on “have I seen this bug?” queries) or for wiki faithfulness. Before I can claim this is a reliable system, I need to go through an extensive process of evals with a smaller dataset.
10. Scholar-digest hallucinations. Auto-triage saves time but sometimes invents citations, datasets, and methods. Weekend review of my collected research materials is a critical part of the system, not optional polish. How can I improve the way this system extracts provenance, methodology, and technical data from papers?

Related

• Build note: /notepad/digital_brain_karpathy_wiki_rag/
• Downstream curriculum use of the same sources: /garden/multi-agent-edu-stack/ (MultiAgentEDUstack, in development)
• Literature-review context for Scholar ingestion: /notepad/week1_literature_review/