Category: systems
-
Crafting Interpreters 第 2 章笔记：A Map of the Territory
TLDR: This note maps the interpreter pipeline from source text through tokens, parsing, semantic analysis, code generation, and runtime choices.
-
Crafting Interpreters 第 3 章笔记：The Lox Language
TLDR: Lox is the small language that carries the book: expressive enough for classes, closures, and control flow, but compact enough to implement twice.
-
Crafting Interpreters 第 4 章笔记：Scanning
TLDR: Scanning is the first hard boundary in an interpreter: raw characters become tokens, and the rest of the language pipeline finally has structure to work with.
-
CS336 Lecture 3：LM Architecture 与 Hyperparameters
TLDR: LM architecture is a stack of trade-offs across normalization, activations, attention, positional encoding, hyperparameters, stability, and inference cost.
-
CS336 Lecture 4：Mixture of Experts
TLDR: MoE scales parameter count through sparse expert routing, but the real work is balancing tokens, capacity, communication cost, and specialization.
-
Compression Is All You Need：把数学进展看成可测压缩
TLDR: Mathematical progress can be viewed as compression when a new abstraction makes many downstream proofs shorter, reusable, or easier to maintain.
-
Heuristic Learning：用代码维护可验证的启发式系统
TLDR: Heuristic Learning frames iterative agent work as maintaining a living heuristic system, where patches, rules, and code are compressed into reusable practice.
-
CS336 Lecture 1：Language Modeling as Engineering
TLDR: Modern LM work is easiest to understand by building the stack yourself, because tokenization, data, compute, and evaluation are all leaky engineering choices.