2026
-
CS336: Lecture 3 - LM Architecture and Hyperparameters
TLDR: LM architecture is a stack of trade-offs across normalization, activations, attention, positional encoding, hyperparameters, stability, and inference cost.
-
CS336: Lecture 4 - Mixture of Experts
TLDR: MoE scales parameter count through sparse expert routing, but the real work is balancing tokens, capacity, communication cost, and specialization.
-
Yuandong Tian talks: search quality is action-space quality
TLDR: More rollouts are not enough. Search becomes powerful when the action space, representation, evaluator, and memory make good trajectories easier to find.
-
Concordia: LLM agents as social simulation actors
Concordia is useful because it treats LLM agents as situated social actors with memory, roles, norms, partial observations, and a world state mediated by a Game Master.
-
Compression Is All You Need: measuring mathematical progress
TLDR: A mathematical abstraction is valuable when it compresses downstream work: proofs become shorter, repeated patterns disappear, and the library becomes easier to extend.
-
Heuristic Learning: maintaining a learning system in code
TLDR: Heuristic Learning treats iterative agent work as maintaining a verifiable software system. Feedback updates code, tests, rules, state representations, and memory rather than neural network weights.
-
自私的基因：第 3 章 不朽的双螺旋
TLDR: The durable unit is not the body but the replicating gene: bodies disappear, while genetic information keeps competing through copying and recombination.
-
CS336: Lecture 1 - Language Modeling as Engineering
TLDR: Modern LM work is easiest to understand by building the stack yourself, because tokenization, data, compute, and evaluation are all leaky engineering choices.