2025-10-31
halloween pts. 1 & 2
Hollis Robbins with more on aphantasia, imagination, and creativity. Something which occurs to me is that many things I consider to be obvious about how LLMs work are not widespread because most people think in either pictures, and are therefore unable to directly “see” in latent space, which means their reasoning about embeddings must be done on fuzzy low-dimensional miniature reconstructions (or to those who think primarily in words, LLMs must seem as if they are black boxes)1.
Huggingface releases the Smol Training Playbook, an in-depth guide to training LLMs.
Robert Long linkthread.
If that’s the case, maybe I should describe my thoughts about how continual learning will be solved, which I previously assumed must be either obvious or obviously wrong.
Essentially, learning is the placement of a new concept within latent space. From reading, experience, or teaching, you place the new idea within an approximate location. From there, you run reinforcement learning to adjust its position such that it maximizes concordance with observation and coherence with your existing worldview. Finally, you run annealing such that the implications of this new idea flow through related concepts, slightly adjusting their positions as well.
This is an oversimplification, because generally speaking, it’s not the case that one token or neuron equals one concept; we probably also use sparse representations when encoding ideas in our heads. In which case, adding a new concept is more akin to identifying all of its relevant relations and adjusting them such that there is an implicit encoding of this new idea. Learning would be obtained by surgically fine-tuning on just these specific relations with high-learning rate, leaving all other weights fixed to prevent catastrophic forgetting, then only afterward one would anneal the entire model with low-learning rate to propagate these changes and improve overall consistency.
In practice, figuring out the relevant relations is hard and surgically fine-tuning is not vector parallelized, so what you would do instead is train a sparse LoRA on the target concept, which is equivalent to bootstrapping this new sparse representation. Then you merge the LoRA into the base model and anneal the result (or even more simply, just distill it instead). What would make it “continual” is a permanent scratchpad LoRA that periodically gets integrated into the base model, along with checkpointing. Presumably, Thinking Machines is even now working on setting up the infrastructure to bring this to the masses.