How GPT, Claude, and Gemini are actually trained and served – Reiner Pope
In this blackboard-style lecture, Reiner Pope walks through how large language models are served in practice, using transformer inference on a GPU cluster to explain why latency and cost behave the way they do. The excerpt focuses on batch size, memory bandwidth, compute throughput, and KV cache fetches, showing how these factors create trade-offs between speed, throughput, and price. It also frames why different serving modes can offer faster token streaming at higher cost.