Posts tagged moe

• CS336: Lecture 4 - Mixture of Experts

  TLDR: MoE scales parameter count through sparse expert routing, but the real work is balancing tokens, capacity, communication cost, and specialization.

  26min read  ·  May 22, 2026

   2026  ·   cs336  ·   moe  ·   learning  ·   systems