Errata for the lecture videos

When a check of the book found an error in a lecture video, the book gives the correct version and the error is listed here. Each time links to the moment in the video. Errors marked serious change a number, a figure or a result that the rest of the lecture relies on. Small slips where the spoken point stands are not listed. A lecture that has been revised links to its second edition, which fixes the errors listed for the first; the first edition is retired, and its list is kept here as a record.

Session 01: Why Language Is Hard (§1.1)

The second edition of this lecture (17 November 2026, watch it) fixes every error listed below for the first edition.

No corrections to the second edition.

First edition (retired)

Time The video says Correct Source
8:56 PP attachment has been a challenge since the 1950s parser work on it starts in the 1970s Kimball 1973; Frazier 1978
10:11 same eight words I saw the man with the telescope has seven words the sentence on screen
10:36 the trophy/suitcase pair is Winograd’s 1972 thesis example it is Levesque, Davis & Morgenstern’s; Winograd’s own example is the council/protesters sentence at 16:22 Levesque et al., KR 2012; Winograd 1972
13:25 a typical NLP vocabulary is 50,000 to 100,000 tokens current models use about 30k (BERT) to 256k (Gemma); Llama 3 uses 128k model papers
13:45 the largest corpora hold about 10^13 sentences about 10^13 tokens, so about 10^12 sentences; the argument only gets stronger Grattafiori et al. 2024
15:10 the Zipf curve shows word frequencies in a real corpus, and half the vocabulary is under 5 per million the curve is illustrative and the ‘half’ figure is unsourced; measured: in 2.1M words of Gutenberg text, 36% of distinct words occur once NLTK Gutenberg count
16:54 the protesters sentence is the same grammar with a different inference its structure differs, so it is not a one-word Winograd pair; Winograd’s own pair swaps feared/advocated Winograd 1972
21:30 cat and feline rarely appear in the same contexts they share contexts (Firth’s point); a count table just cannot link them Firth 1957

Session 02: Words as Vectors (§1.2)

The second edition of this lecture (19 November 2026, watch it) fixes every error listed below for the first edition.

No corrections to the second edition.

First edition (retired)

Time The video says Correct Source
1:26 the word map is a learned embedding, flattened it is a hand-placed schematic, not computed from a trained model deck generator
8:06 king - man + woman comes from the NeurIPS 2013 word2vec paper it comes from Mikolov, Yih & Zweig, NAACL 2013 Mikolov et al. 2013, arXiv 1301.3781
18:18 t-SNE keeps distances inside a cluster t-SNE keeps neighbourhoods; distances and cluster sizes are not reliable van der Maaten & Hinton 2008
20:26 rare words: skip-gram handles well, GloVe less reliable the GloVe paper does compare them: on its Rare Words test GloVe scores 38.1, skip-gram 37.2, CBOW 32.5 (6B tokens), so GloVe is not less reliable Pennington et al. 2014, Table 3

Session 03: Sequences and Memory (§2.1)

The second edition of this lecture (21 November 2026, watch it) fixes every error listed below for the first edition.

No corrections to the second edition.

First edition (retired)

Time The video says Correct Source
3:39 every language model before 2017 was recurrent n-gram models and Bengio et al.’s neural LM were not Bengio et al., JMLR 2003
9:49 the condition is |W_h| < 1 it is on the largest singular value of W_h Pascanu et al. 2013
10:25 Bengio 1994 proved the RNN fundamentally broken for long sequences they showed a trade-off between stable memory and gradient learning Bengio, Simard & Frasconi 1994
12:46 bidirectional LSTMs, 2015 bidirectional RNNs date from 1997 Schuster & Paliwal 1997
14:03 the gradient of the addition is just 1, so the cell carries information unchanged with a forget gate it is f_t, so memory decays as the product of the f_t; the forget gate is Gers et al. 2000, not the 1997 LSTM Gers et al. 2000
15:19 the LSTM diagram puts the output gate on the cell-state line C_t passes on untouched and h_t = o_t x tanh(C_t), as the formula on screen says Hochreiter & Schmidhuber 1997
20:15 LSTMs and GRUs were both defaults from 1997 to 2017 the GRU dates from 2014 and LSTMs became the NLP default around 2013-14 Cho et al. 2014
23:09 LSTMs did not benefit from larger models or more data no source supports this; treat it as opinion none found

Session 04: The Attention Breakthrough (§2.2)

The second edition of this lecture (14 November 2026, watch it) fixes every error listed below for the first edition.

No corrections to the second edition.

First edition (retired)

Time The video says Correct Source
3:40 in the alignment heatmap, assis, sur and le peak on on, the, mat serious they should peak on sat, on, the; those rows are shifted one column Le chat s’est assis sur le tapis
6:20 before Bahdanau, alignments were defined by hand statistical MT already learned them without supervision; new was learning them jointly with translation Brown et al. 1993; Bahdanau et al. 2015
8:14 softmax of scores 0.72, 0.20, 0.08 gives 0.55, 0.32, 0.13 serious it gives 0.47, 0.28, 0.25 arithmetic
20:05 Bahdanau-era attention is parallelisable only self-attention is; Bahdanau’s model still runs an RNN step by step Bahdanau et al. 2015
20:05 long-range dependencies are preserved exactly attention gives a direct path, not exact preservation: the weights are a learned softmax Vaswani et al. 2017
21:12 scramble the word order and the weights are identical they are permuted along with the words, not identical arithmetic

Session 05: The Transformer (§3.1)

Time The video says Correct Source
8:18 the model learns to read position from these signals, and it does Vaswani et al. state this as a hypothesis Vaswani et al. 2017
10:45 the two heads show real numbers the head weights are hand-picked for illustration, not fitted deck generator
12:46 the FFN layers store most of the model’s factual knowledge Geva et al. show FFNs act as key-value memories; they measure no share of factual knowledge Geva et al. 2021
13:29 residual connections fix gradients that vanish He et al. address degradation, which they say is not caused by vanishing gradients He et al. 2016
14:32 layer norm normalises one sequence at a time it normalises each position across its features Ba et al. 2016
16:50 depth is most of what ‘a bigger model’ has meant GPT-3 grew width 24x and depth 16x; loss depends only weakly on shape Kaplan et al. 2020
21:38 training is faster by orders of magnitude about 7x fewer training FLOPs than GNMT+RL Vaswani et al. 2017, Table 2

Session 06: BERT: Reading the Whole Sentence (§3.2)

Time The video says Correct Source
13:00 16 TPUs, each square one Cloud TPU BERT-Base used 4 Cloud TPUs, 16 chips, BERT-Large 16 Cloud TPUs, 64 chips Devlin et al. 2019, App. A.2
15:44 prior best bars, and every BERT bar is taller serious prior best was GLUE 72.8, SQuAD 1.1 91.7, SQuAD 2.0 78.0, SWAG 78.0, NER 93.1; BERT-Large’s NER 92.8 did not beat it Devlin et al. 2019, Tables 2-4, 7
16:44 model size and data did not change, so bidirectionality not scale BERT-Large is about 3x GPT and used more data; only the BERT-Base ablation controls this Devlin et al. 2019
17:54 DistilBERT is 60% smaller and 40% faster 40% smaller, 60% faster, 97% of the performance Sanh et al. 2019
18:16 ALBERT has 18x fewer parameters than BERT-Large with competitive performance the 18x model (ALBERT-large) scores clearly lower; the competitive one is the 235M ALBERT-xxlarge Lan et al. 2020

Session 07: GPT: Predicting What Comes Next (§3.3)

Time The video says Correct Source
0:59 Claude, GPT, Gemini, LLaMA, DeepSeek are all decoder-only stacks trained on next-token prediction (again at 25:13) Claude’s and GPT-4’s architectures are unpublished, and all of them are also post-trained with SFT and RLHF model reports
9:23 in post-norm stacks the gradients start to vanish post-norm has large gradients near the output at initialisation, hence warm-up Xiong et al. 2020
25:13 decoder-only is the only branch where the task lives in the text T5, an encoder-decoder, also writes every task as text; what is particular to decoder-only models is that they are trained only to continue the text Raffel et al. 2020
18:12 greedy decoding on a repeating context essentially always loops a loop is forced only with a fixed window; with a growing prefix it is an empirical tendency Holtzman et al. 2020
24:09 Canberra and Sydney score the same on everything the objective measures the objective scores P(next | prefix), which favours Canberra; the point that there is no truth term stands Lin et al. 2022, TruthfulQA

Session 08: Scaling Laws and Emergence (§4.1)

Time The video says Correct Source
1:32 GPT-3’s run drawn to scale: three and a half months on thousands of GPUs OpenAI published neither the wall-clock time nor the GPU count, only V100s on a Microsoft cluster Brown et al. 2020
4:49 six real runs six points sampled from Kaplan’s fitted law, so on the line by construction deck generator
9:10 GPT-4’s loss followed exactly the power law derived here, predicted before it existed the fit adds an irreducible-loss term, L(C) = aC^b + c, and the prediction was made shortly after the run began OpenAI 2023
15:04 Kaplan’s fixed schedule penalised data-heavy runs that ran past it serious Kaplan used one cosine schedule to 130B tokens and read shorter runs from intermediate checkpoints, which overestimates their loss; no run outlasted its schedule Hoffmann et al. 2022, Sec. 2
20:18 four runs over three decades of compute, all at zero they span one decade (2.3e20 to 2.3e21 FLOPs) and score 0.15 to 0.55% Brown et al. 2020, Table H.1
21:14 one MMLU curve it joins GPT-3 175B (43.9) and Gopher (60.0); Wei et al. plot each family separately Wei et al. 2022

Session 09: The Model Zoo (§4.2)

Time The video says Correct Source
5:09 Claude 4 placed in early 2025 that date is Claude 3.7 Sonnet (24 Feb 2025); Claude 4 came out 22 May 2025 Anthropic
9:58 InstructGPT used thousands of labellers about 40 contractors Ouyang et al. 2022
10:55 GPT-4V is a vision encoder bolted on afterwards OpenAI has not published how GPT-4’s vision was built OpenAI 2023
13:21 LLaMA 1 and 2 could just be downloaded LLaMA 1 went only to approved researchers under a non-commercial licence; Llama 2 was openly downloadable Meta 2023
13:30 LLaMA 1 and 2 use standard multi-head attention; LLaMA 3 adds GQA and RoPE serious RoPE was in LLaMA 1, GQA in Llama 2 34B/70B; Llama 3 uses GQA at every size Touvron et al. 2023a, 2023b
15:11 RoPE lets a model trained on 8k tokens make sense of token 50k RoPE alone does not let a model work past its training length; Llama 3 went from 8k to 128k tokens by continued pre-training on long text with RoPE base 500,000 Press et al. 2022; Grattafiori et al. 2024
16:01 DeepSeek entered in 2024 DeepSeek LLM and DeepSeek Coder came out in November 2023 DeepSeek-AI 2023
16:36 GRPO needs no reward model and no human-written reasoning examples (also 23:12, 25:09) serious GRPO drops PPO’s learned value model (the critic), not the reward model: DeepSeekMath still scored its samples with a learned reward model. Rule-based rewards and no supervised reasoning data are DeepSeek-R1-Zero only; the released R1 used cold-start chain-of-thought data Shao et al. 2024; DeepSeek-AI 2025
17:21 MLA throws nothing away, it only compresses the 512-dim latent is a real low-rank limit; it works because the model is trained with it DeepSeek-V2 paper, 2024
17:53 DeepSeek routes each token to the top two experts DeepSeek-V2 uses 6 of 160 routed experts plus 2 shared; top-2 of 8 is Mixtral’s scheme DeepSeek-V2 paper, 2024
20:33 Gemini Ultra’s 90.0 on MMLU, beside 5-shot scores 90.0 is CoT@32; Ultra’s 5-shot score is 83.7 Gemini Team 2023

Session 10: Training and Alignment (§5.1)

Time The video says Correct Source
2:32 look at what the raw model gives back these outputs are constructed examples, not recorded ones deck generator
3:29 ChatGPT runs on the same family of weights as GPT-3; 100 million users in two months it was fine-tuned from a GPT-3.5-series model; the 100M figure is a UBS estimate OpenAI 2022; Reuters 2023
7:47 FLAN used a few thousand demonstrations FLAN rewrote 60+ existing NLP datasets through templates; written demonstrations are InstructGPT’s Wei et al. 2022; Ouyang et al. 2022
18:24 Constitutional AI labels (original, revised) as the preference pair serious revisions are supervised targets; preference labels come from a model comparing two samples against a principle, and human helpfulness labels are kept Bai et al. 2022
18:46 Constitutional AI trains every Claude model since no primary source states this for every model none found
22:02 the group baseline replaces both the reward model and the value network (also 23:19, 24:46) serious GRPO drops PPO’s learned value model (the critic), not the reward model: DeepSeekMath still scored its samples with a learned reward model. Rule-based rewards and no supervised reasoning data are DeepSeek-R1-Zero only; the released R1 used cold-start chain-of-thought data Shao et al. 2024; DeepSeek-AI 2025
28:23 SFT fills helpful only; harmless and honest live only in RL SFT data include refusals and safety examples, and RLHF’s honesty gain is partial Ouyang et al. 2022; Touvron et al. 2023b
29:22 what GRPO takes out of DPO GRPO takes the value model out of PPO and keeps on-policy sampling, which DPO drops Shao et al. 2024

Session 11: Making Models Efficient (§5.2)

Time The video says Correct Source
12:14 trained at 8k, serving at 128k thanks to RoPE (also 5:01, 36:46) serious RoPE alone does not let a model work past its training length; Llama 3 went from 8k to 128k tokens by continued pre-training on long text with RoPE base 500,000 Press et al. 2022; Grattafiori et al. 2024
21:09 Mixtral, gpt-oss and DeepSeek-V3 are all an order of magnitude below the line Mixtral is only 3.6x below, with 12.9B of its 46.7B parameters active Jiang et al. 2024
27:13 the best published 2-bit result is 6.55 6.55 is OmniQuant’s g128 row (g64: 6.11); AQLM reaches 3.94 at 2.07 bits Shao et al. 2024; Egiazarian et al. 2024
35:07 +0.11 perplexity, measured, on the Llama 3 70B ledger the 0.11 was measured on Llama 2 70B, with GPTQ at group size 128 OmniQuant, Table 1

Session 12: Prompting, RAG, and Agents (§6.1)

Time The video says Correct Source
6:56 the prompt is the left-hand side of the bar the predicted token is left of the bar; the prompt sits between the bar and the semicolon equation on screen
10:48 ‘think step by step’, five words in the prompt, gives these gains (also quiz at 11:38) serious these PaLM numbers are few-shot chain of thought with 8 worked examples; zero-shot ‘Let’s think step by step’ is Kojima et al., GSM8K 10.4 to 40.7 Wei et al. 2022; Kojima et al. 2022
15:25 the embedding space from session 2, unchanged RAG retrieves with trained passage encoders (DPR), not word vectors Lewis et al. 2020
15:45 the 3 smallest angles to the question the drawing picks nearest by distance; the two agree only for unit vectors deck generator
18:40 the whole drop from 88.3 to 53.8 is caused by placement moving the document first recovers 22.0 points (75.8); the rest is the 19 distractors Liu et al. 2024
25:45 fine-tuning can add facts it never saw retrieval beats fine-tuning for new knowledge, and new facts learned this way raise hallucination Ovadia et al. 2024; Gekhman et al. 2024

Session 13: Hallucination and Trust (§7.1)

Time The video says Correct Source
3:39 the distribution is the same shape for Kiknadze as for Paris (recap at 31:12 also prints Kandelaki) serious the distributions differ (long-tail ones are flatter, as the later calibration and self-consistency slides show); what we cannot see is the probability of the sampled token. The name is Kiknadze GPT-4 report (OpenAI 2023), Fig. 8
12:09 the error rate is at least the singleton rate, a floor (also 14:04) it is a bound with slack terms that holds with 99% probability for a calibrated model that answers everything; it is vacuous for small N and abstention escapes it Kalai et al. 2025, Theorem 2
15:25 guessing wins by more the less the model knows answering beats abstaining by a margin of p, so the incentive to guess is smallest where the model knows least Kalai et al. 2026
16:46 every one replaced a benchmark that had saturated unsourced; IFEval and SWE-bench had no saturated predecessor Kalai et al. 2025, Table 2

Session 14: Ethics, Bias, and What We Do Not Know (§7.2)

Time The video says Correct Source
1:54 Amazon’s CV tool was scrapped before it reached hiring recruiters looked at its rankings but never relied solely on them; the team was then disbanded Reuters, Dastin 2018
2:21 the helpline chatbot gave weight-loss advice to people who called for help the documented case is an activist, Sharon Maxwell, testing it NPR 2023
2:44 TOEFL essays written under supervised exam conditions; the detector is still marking student work the essays came from a Chinese education forum; ‘still marking’ is unsourced Liang et al. 2023
6:07 every other language on the web comes to 36.7%, so English is bigger than all of them combined 36.7% is the languages after the first five; all languages other than English hold 59.5% against English’s 40.5% Common Crawl CC-MAIN-2026-34
12:35 Gemini applied a diversification step to prompts Google spoke of tuning; the exact mechanism is unpublished Google, Raghavan 2024
13:43 Kenyan annotators labelled data for preference rankings they labelled toxic text for a ChatGPT safety detector Time, Perrigo 2023
17:25 same tool, same accuracy overall 63.8% for Black and 67.0% for white defendants; Northpointe argued predictive parity ProPublica 2016
24:17 that work runs on small open models the 2025 studies traced Claude 3.5 Haiku; only the open library targets small open models. The on-screen “a few million features, named”: about 34 million were learned and a sample named Lindsey et al. 2025
25:27 MMLU arrives in 2021 released September 2020, published at ICLR 2021 Hendrycks et al. 2021