Category: learning
-
Sutton RL: Day 1 - RL Problem and MDP Basics
TLDR: RL is interaction for long-term reward: policy chooses actions, reward gives feedback, value estimates future return, and Bellman equations connect the pieces.
-
Crafting Interpreters: Chapter 2 - A Map of the Territory
TLDR: This note maps the interpreter pipeline from source text through tokens, parsing, semantic analysis, code generation, and runtime choices.
-
Crafting Interpreters: Chapter 3 - The Lox Language
TLDR: Lox is the small language that carries the book: expressive enough for classes, closures, and control flow, but compact enough to implement twice.
-
Crafting Interpreters: Chapter 4 - Scanning
Scanning is the first structural boundary in an interpreter: raw characters become tokens, so the parser can work with language units instead of individual bytes.
-
CS336: Lecture 3 - LM Architecture and Hyperparameters
TLDR: LM architecture is a stack of trade-offs across normalization, activations, attention, positional encoding, hyperparameters, stability, and inference cost.
-
CS336: Lecture 4 - Mixture of Experts
TLDR: MoE scales parameter count through sparse expert routing, but the real work is balancing tokens, capacity, communication cost, and specialization.
-
CS336: Lecture 1 - Language Modeling as Engineering
TLDR: Modern LM work is easiest to understand by building the stack yourself, because tokenization, data, compute, and evaluation are all leaky engineering choices.
-
CS336: Lecture 2 - PyTorch and resource accounting
Lecture 2 is about making training cost concrete: tensors, dtypes, memory, FLOPs, autograd, optimizers, data loading, checkpoints, and mixed precision all have resource prices.