How are large language models trained?
Google explains LLM pre-training as self-supervised next-token prediction at massive engineering scale.
“pre-training is doing this at scale. This phase of LLM training is a massive engineering challenge.”
A Google Developers educational video explains LLM training steps, covering pre-training via next-token prediction and the engineering challenges of distributing models across thousands of GPUs. The content is introductory and does not contain new announcements, research findings, or industry signals. It briefly mentions post-training alignment but cuts off before substantive detail.