THE LEARNING PATH
Learn the mechanisms, step by step.
Guided lessons connect concepts, Lab experiments and chapters from both books. Start with a concrete situation, build the reasoning, then verify it.
Beginner7 min readVolume I
Before any matrix multiplication, text becomes a sequence of identifiers. Try a BPE tokenizer and see the effect on cost.
tokenizationBPEcostVolume I
Read the lesson : Splitting text into tokens determines cost and latency →Beginner7 min readVolume I
Embeddings place words in a space where relationships become translations. Manipulate them in a 3D observatory.
embeddingslatent spacecosine similarityVolume I
Read the lesson : The geometry of latent space turns meaning into vectors →Intermediate8 min readVolume I
Queries, keys and values: see how each token queries the others, and why we divide by √d_k before the softmax.
attentionTransformersoftmaxVolume I
Read the lesson : Multi-head attention links each word to its context in parallel →Intermediate8 min readVolume I
Rather than adding a position, RoPE rotates Queries and Keys. Test on a dial why only the gap matters.
RoPEpositionlong contextVolume I
Read the lesson : RoPE encodes relative distance through a geometric rotation →Intermediate8 min readVolume I
A router activates only a few experts per token. Simulate Top-K routing and collapse, and tell total parameters from active ones.
MoEroutingMixtralVolume I
Read the lesson : Mixture of Experts specializes computation without weighing down inference →Advanced9 min readVolume I
The real bottleneck is memory, not compute. Compare HBM and SRAM traffic, then estimate a model’s VRAM with a sizing tool.
FlashAttentionKV cacheVRAMVolume I
Read the lesson : FlashAttention and the KV cache tame the GPU memory hierarchy →Intermediate12 min readVolume II
Learn how features, circuits and causal interventions help investigate an LLM—and what these methods still cannot prove.
mechanistic interpretabilitytransformer circuitscausal intervention
Read the lesson : Mechanistic interpretability of LLMs: a practical guide →Intermediate7 min readVolume II
Is information visible inside a model actually used? Build a contrast, measure restoration and learn how to test an explanation.
activation patchinglogits
Read the lesson : The LLM garage: patch an activation to understand →Beginner9 min readVolume II
Follow a request through five checks: intent, extraction, evidence, generation and evaluation.
evaluationRAGreliability
Read the lesson : From question to verifiable answer →