Large Language Models: How They Grew, One Problem at a Time

Draft

Author

Vahid Seydi

Preface

Cover: islands growing along a voyage route, one problem at a time

This is the draft of Large Language Models: How They Grew, One Problem at a Time, the companion book to Large Language Models, a free lecture course from Bluevision AI. Each chapter covers a run of consecutive lectures, and each section covers one lecture: section 1.1 is Session 01, and so on. The lectures give each idea as a picture and in speech; this book gives the full argument, the references and the exercises.

The title names the method of the book, which we call the problem chain. Each section starts from a problem that the previous section left open, solves it, and ends on the new problem that the solution creates. Language is hard for a machine to represent, so we turn words into vectors. Vectors ignore word order, so we read text as a sequence. Recurrent networks forget the start of a long sequence, so we add attention. Attention on its own becomes the Transformer, and the chain continues through scale, alignment, efficiency and the limits of what these models can be trusted to do. The chain follows, roughly, the order in which the field itself grew, so it is also a short history of how large language models came to be.

About this draft

This is a review draft. Every section has been checked for facts, mathematics and references, and tested on readers new to the subject, but it has not yet been through outside review. If you find an error, or a passage that does not make sense, please use the Report an issue link on the page; a free GitHub account is enough. Corrections are fixed in the online edition. Errors in the published lecture videos are listed on the Errata page.

Authorship and the use of AI

Years of research and teaching across computer engineering, software engineering and artificial intelligence, combined with a broad understanding of machine learning and its many different approaches, have shaped my vision of how these subjects should be taught.

As an author and designer of educational content, I wanted to develop a structured learning plan that reflects what I believe to be an effective approach to teaching artificial intelligence and machine learning. My aim has been to turn this vision into engaging, coherent and well-designed educational materials.

To achieve this, I drew on the capabilities of agentic AI systems. I presented my ideas to a range of AI agents that I personally designed, trained and developed, and worked with them to implement the ideas and teaching approaches reflected in the materials you see here. In the lecture videos, the voice you hear is synthetic, and it is an AI agent speaking.

The result is a collaboration between me and a group of AI agents: my AI students, rather than human students. Of course, the many real students I have had the privilege of teaching over the years have also played an important role in shaping this work. Their questions, challenges, curiosity and enthusiasm for learning have all fed my passion for teaching and my continuing desire to learn, improve and find better ways of communicating knowledge.

The AI systems used here are tools. They are not authors and are not listed as authors. I am responsible for every statement in the book, including its errors.

About the author

Vahid Seydi is a Senior Lecturer in Data Science/Data Analytics at UWE Bristol and an Honorary Research Fellow at Bangor University’s School of Ocean Sciences. From mathematics and software engineering to artificial intelligence, he has more than 20 years of experience, and his work, grounded in the mathematical foundations of machine learning, has followed the field’s own development: from neural networks, fuzzy systems and evolutionary optimisation, through probabilistic models, reinforcement learning and transfer learning, to deep learning, transformers, large language models and diffusion models. Before writing this book he studied how machine learning is taught at more than 40 universities, and the plan of the book comes from that work.

How to cite this book

Cite the book, or the chapter you used. Please do not cite the videos: they point here.

Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft). Bluevision AI. https://books.bluevision-ai.org/LLM/

@misc{seydi2026LLM,
  author    = {Seydi, Vahid},
  title     = {Large Language Models: How They Grew, One Problem at a Time},
  year      = {2026},
  note      = {Draft},
  publisher = {Bluevision AI},
  url       = {https://books.bluevision-ai.org/LLM/}
}

To cite one chapter:

  • Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft), Chapter 1: Representing Language. Bluevision AI. https://books.bluevision-ai.org/LLM/
  • Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft), Chapter 2: Sequences and Attention. Bluevision AI. https://books.bluevision-ai.org/LLM/
  • Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft), Chapter 3: The Transformer and Its Descendants. Bluevision AI. https://books.bluevision-ai.org/LLM/
  • Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft), Chapter 4: Scale. Bluevision AI. https://books.bluevision-ai.org/LLM/
  • Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft), Chapter 5: Training, Alignment and Efficiency. Bluevision AI. https://books.bluevision-ai.org/LLM/
  • Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft), Chapter 6: Using a Frozen Model. Bluevision AI. https://books.bluevision-ai.org/LLM/
  • Seydi, V. (2026). Large Language Models: How They Grew, One Problem at a Time (draft), Chapter 7: Trust and Its Limits. Bluevision AI. https://books.bluevision-ai.org/LLM/