01
Learn →
Progressive lessons to build understanding.
UNDERSTAND · EXPERIMENT · VERIFY
See what happens inside an LLM (large language model).
From the first token to the mechanisms behind an answer.
The model in motion
Read the film description
Follow a sentence through tokens, vectors, attention, Transformer layers and next-token selection.
From text to numbers. The sentence “The capital of France is” splits into five words, treated here as tokens. Each token becomes a vector; twelve dimensions are illustrated in a schematic space.
From relations to context. Connections between positions become a matrix of attention weights. Each row attends to earlier positions and its own position. Future positions are masked.
Through the layers. The representation passes through the Transformer layers. Attention, the MLP and the residual stream contribute to its transformation. The journey through the layers is schematic.
From scores to the next token. The final position produces logits for four candidates: Paris 8.7; Lyon 3.1; Marseille 1.8; London 0.9. Softmax at temperature 1 converts them to probabilities totaling 100%. Paris receives about 99.49% in this reduced example. The most probable token joins the sentence: “The capital of France is Paris”. The process can repeat.
Educational illustration: simplified word-level tokenization, fictional vectors and weights. Probabilities are calculated over four candidates, not a model’s full vocabulary. The film selects the maximum; models can also sample.
Understand, experiment, then follow new developments
02
Lab →
Guided experiments to test an idea.
03
Articles →
Lasting analyses to explore further.
04
News →
AI developments and their uses.
Visual explanations
Clear, progressive explanations.
Interactive experiments
Change something. Observe its effects.
From theory to practice
Real situations and results to verify.
One connecting thread
Two books to take you further.
Two books. One connecting thread.
Understand the machine. Then learn how to investigate it.
VOLUME I / UNDERSTAND
The Mechanics of LLMs
From language to architectures and applications: build a sound understanding of what an LLM does.
- Tokens and representations
- Attention and Transformer architecture
- Training and generation
- RAG, agents and evaluation
VOLUME II / INVESTIGATE
The Internal Mechanics of LLMs
Go beyond appearances: measure, inspect and intervene to examine what the results really let you conclude.
- Representations and probes
- Inspection and interventions
- Controlled experiments
- Evidence and limits of conclusions
Learn
A model, step by step.
How does a sentence become a proposed answer? Select a step to follow the computation.
Key idea
It starts with context.
You write “The capital of France is”. The model receives this context and proposes what follows. It does not choose a whole answer at once: here we examine one generation step.
Educational diagram: illustrative tokenisation, vectors, weights and probabilities. No model is executed.
Learn
Learn the mechanisms, step by step.
Guided lessons connect concepts, Lab experiments and chapters from both books. Start with a concrete situation, build the reasoning, then verify it.
The LLM garage: patch an activation to understand
Is information visible inside a model actually used? Build a contrast, measure restoration and learn how to test an explanation.
Read the lesson : The LLM garage: patch an activation to understand →From question to verifiable answer
Follow a request through five checks: intent, extraction, evidence, generation and evaluation.
Read the lesson : From question to verifiable answer →The laboratory / 01—05
A good answer deserves an experiment.
Imagine an assistant handling customer requests. At each step, a plausible answer can hide an error. These five experiments help you build useful checks.
0102030405
Understand the request
Distinguish success on a dataset from the ability to recognize the problem that matters.
Extract the right information
Understand why usable formatting and correct information are separate requirements.
Find the applicable policy
Distinguish an apparently relevant document from sufficient evidence for a specific situation.
Understand variation in answers
Distinguish the probability of selecting a token from the probability that an answer is true.
Evaluate the service provided
Read precision from the perspective of received alerts and recall from the perspective of actual incidents.
Articles
Take time to understand.
Detailed explanations, examples and methods to extend your experiments.
Precision and recall: choose what to protect
A needless alert and a missed incident may have very different costs.
Read the article →A perfect score deserves an investigation
Build a test that distinguishes useful ability from an accidental shortcut.
Read the article →Temperature does not control truth
Understand what decoding changes, and what it cannot guarantee.
Read the article →Three checks before trusting an extraction
Clean JSON can contain the wrong information. Separate form from meaning with a concrete protocol.
Read the article →News
AI news
New models, research and tools: what’s new in artificial intelligence, how it works and how it is used.
Models
Published
Claude Sonnet 5.5: Anthropic focuses on speed
Anthropic introduces a new Sonnet version, highlighting faster execution and lower token consumption.
Read the update : Claude Sonnet 5.5: Anthropic focuses on speed →Tools and uses
Published
Mistral joins Firefox Smart Window
Mozilla and Mistral announce a partnership for Smart Window, Firefox’s browsing assistant, currently in beta.
Read the update : Mistral joins Firefox Smart Window →Methods and architectures
Published
Mistral Small 4: how a mixture of experts works
A look back at Small 4’s architecture: why can a model contain many experts without activating them all for every token?
Read the update : Mistral Small 4: how a mixture of experts works →