How did we get here

A reading list of the papers that, step by step, led from the idea of a computing machine to language models and autonomous robots.

Evolution, paper by paper

• 1936.05: On Computable Numbers, with an Application to the Entscheidungsproblem

  

  

  Alan Turing. Defines computation itself: a universal machine that runs any program handed to it as data.

• 1943.12: A Logical Calculus of the Ideas Immanent in Nervous Activity

  

  

  Warren McCulloch, Walter Pitts. The first mathematical neuron: weighted inputs, a threshold, a binary output. Networks of such units compute any logical function. The artificial neuron starts here.

• 1945.06: First Draft of a Report on the EDVAC

  

  

  John von Neumann. The stored-program architecture: program and data in one memory, one processor that fetches and executes. Most personal computers are still built on the von Neumann architecture.

• 1948.07: A Mathematical Theory of Communication

  

  

  Claude Shannon. Information theory: the bit, entropy, channel capacity. Cross-entropy loss and perplexity, the numbers we watch while training, come from this paper. It also contains the first n-gram language models of English text.

• 1958.11: The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain

  

  

  Frank Rosenblatt. The first machine that learned its weights from examples instead of being programmed. A linear classifier, and the ancestor of every layer in a modern network.

• 1969.08: Gaussian elimination is not optimal

  

  

  Volker Strassen. Multiplies 2x2 blocks with 7 multiplications instead of 8, so n x n matrices cost O(n^2.81) instead of n^3. The first proof that the textbook algorithm is not the end of the story, and a record that stood for the small cases for half a century.

• 1986.10: Learning representations by back-propagating errors

  

  

  David Rumelhart, Geoffrey Hinton, Ronald Williams. Backpropagation for multi-layer networks, made practical and shown to learn useful hidden representations. The algorithm every training loop still runs.

• 1988.08: Learning to Predict by the Methods of Temporal Differences

  

  

  Richard Sutton. Learn from the difference between successive predictions instead of waiting for the final outcome. The core update rule of reinforcement learning, from TD-Gammon to AlphaGo’s value network.

• 1992.05: Q-learning

  

  

  Christopher Watkins, Peter Dayan. Learn the value of every action in every state from experience alone, with no model of the environment, and a proof that it converges. The algorithm DQN later put a deep network under.

• 1992.05: Simple statistical gradient-following algorithms for connectionist reinforcement learning

  

  

  Ronald Williams. Policy gradient, REINFORCE: move the network’s weights directly in the direction that makes rewarded actions more likely. The ancestor of PPO and of the RL step inside RLHF.

• 1995.03: Temporal Difference Learning and TD-Gammon

  

  

  Gerald Tesauro, IBM. A neural network that learned backgammon by playing against itself with TD learning and reached the level of the best humans. The first proof that self-play plus a neural value function works.

• 1997.05: Deep Blue

  

  

  Murray Campbell, Joseph Hoane, Feng-hsiung Hsu, IBM. Beat Garry Kasparov in May 1997 with brute-force search, 200 million positions per second, and a hand-tuned evaluation function; the paper describing it appeared in 2002. The peak of the no-learning approach, and the benchmark AlphaZero later made obsolete.

• 1997.11: Long Short-Term Memory (LSTM)

  

  

  Sepp Hochreiter, Jürgen Schmidhuber. Gates and a memory cell let gradients survive hundreds of steps, solving the vanishing-gradient problem. The dominant sequence model for twenty years, until the transformer.

• 2003.03: A Neural Probabilistic Language Model

  

  

  Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin. A neural network that predicts the next word from learned word vectors. The embedding table plus next-token objective that GPT still uses was born here.

• 2013.10: Word2Vec (Distributed Representations of Words and Phrases and their Compositionality)

  

  

  Tomas Mikolov et al., Google. Cheap, scalable word embeddings trained with negative sampling; vector arithmetic like king - man + woman ≈ queen. Made distributed representations a standard tool.

• 2013.12: Playing Atari with Deep Reinforcement Learning

  

  

  Volodymyr Mnih et al., DeepMind. Q-learning with a convolutional network reading raw pixels, plus experience replay and a target network: one architecture for 49 Atari games (Nature version, February 2015). Deep learning and reinforcement learning joined.

• 2014.09: Seq2Seq (Sequence to Sequence Learning with Neural Networks)

  

  

  Ilya Sutskever, Oriol Vinyals, Quoc Le, Google. Encoder-decoder with LSTMs: read a whole sentence into a vector, generate another sentence from it. End-to-end machine translation, and the encoder-decoder pattern the transformer later inherited.

• 2015.05: Highway Networks

  

  

  Rupesh Srivastava, Klaus Greff, Jürgen Schmidhuber. Gated shortcut paths that let networks hundreds of layers deep train at all. The idea residual connections simplified seven months later.

• 2015.12: Deep Residual Learning for Image Recognition

  

  

  Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Microsoft Research. The residual connection: learn the difference and add it to the input. Made 152-layer networks trainable; every transformer block carries two of these shortcuts.

• 2016.01: Mastering the game of Go with deep neural networks and tree search

  

  

  David Silver et al., DeepMind. Policy and value networks trained on human games and self-play, guiding Monte Carlo tree search. AlphaGo beat Lee Sedol 4-1 in March 2016, a decade earlier than expected.

• 2017.06: Attention is All You Need

  

  

  Ashish Vaswani et al., Google. The transformer: self-attention instead of recurrence, so a whole sequence trains in parallel. The architecture of GPT, BERT and nearly everything since.

• 2017.07: Proximal Policy Optimization Algorithms

  

  

  John Schulman et al., OpenAI. A policy-gradient method that clips each update so the policy cannot move too far at once. Simple, stable, and the default RL optimizer, including inside RLHF.

• 2017.12: Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

  

  

  David Silver et al., DeepMind. No human games, no hand-made evaluation: AlphaZero learns from self-play starting at random weights, one algorithm for chess, shogi and Go (Science, December 2018). Twenty years after Deep Blue, search plus a learned network with zero human knowledge beat the strongest engines.

• 2018.06: GPT-1 (Improving Language Understanding by Generative Pre-Training)

  

  

  Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, OpenAI. A decoder-only transformer pretrained to predict the next token on BooksCorpus, then fine-tuned per task. 117M parameters, about 1B tokens (800M words, 7,000 books).

• 2018.10: BERT (BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding)

  

  

  Jacob Devlin et al., Google. An encoder-only transformer trained on masked tokens, reading in both directions. Dominated understanding tasks and made pretraining plus fine-tuning the default for NLP.

• 2019.02: GPT-2 (Language Models are Unsupervised Multitask Learners)

  

  

  Alec Radford et al., OpenAI. The same architecture scaled up: 1.5B parameters, 40 GB of WebText, about 10B tokens. Solved tasks it was never trained for, zero-shot.

• 2020.05: GPT-3 (Language Models are Few-Shot Learners)

  

  

  Tom Brown et al., OpenAI. 175B parameters, 300B tokens. In-context learning: show a few examples in the prompt and the model does the task with no weight update. Scaling became the strategy.

• 2020.12: Mastering Atari, Go, chess and shogi by planning with a learned model

  

  

  Julian Schrittwieser et al., DeepMind. MuZero plans inside a model of the environment that it learned itself, without being told the rules. Model-based RL reaches AlphaZero’s level.

• 2021.03: CLIP (Contrastive Language-Image Pre-training)

  

  

  Alec Radford, Jong Wook Kim et al., OpenAI. Image and text encoders trained contrastively on 400M pairs to share one embedding space. Zero-shot image classification, and the bridge later multimodal models are built on.

• 2021.07: Highly accurate protein structure prediction with AlphaFold

  

  

  John Jumper et al., DeepMind. Predicts a protein’s 3D structure from its amino-acid sequence to near-experimental accuracy, with attention over residues and residue pairs. The fifty-year-old protein folding problem was effectively solved; the 2024 Nobel Prize in Chemistry followed.

• 2022.03: RLHF (Training language models to follow instructions with human feedback / InstructGPT)

  

  

  Long Ouyang et al., OpenAI. Supervised fine-tuning, a reward model learned from human preferences, then reinforcement learning. Turns a next-token predictor into an assistant that follows instructions; the recipe behind ChatGPT.

• 2022.04: Flamingo: a Visual Language Model for Few-Shot Learning

  

  

  Jean-Baptiste Alayrac et al., DeepMind. A frozen language model and a frozen vision encoder joined by a few trained layers. Few-shot learning over interleaved images and text; the first strong visual language model.

• 2022.10: Discovering faster matrix multiplication algorithms with reinforcement learning

  

  

  Alhussein Fawzi et al., DeepMind. AlphaTensor turns the search for a matrix-multiplication algorithm into a single-player game and plays it with AlphaZero-style RL. Found 47 multiplications for 4x4 matrices over GF(2), beating Strassen’s 49 that had stood since 1969. An AI improved an algorithm the whole field runs on.

• 2023.01: BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

  

  

  Junnan Li et al., Salesforce. A small Q-Former bridges a frozen image encoder and a frozen LLM at a fraction of the training cost. You do not have to retrain the big models to make them see.

• 2023.03: PaLM-E: An Embodied Multimodal Language Model

  

  

  Danny Driess et al., Google. Robot sensor readings and images become tokens inside a 562B language model, which then plans actions. Language models enter robotics.

• 2023.04: LLaVA: Visual Instruction Tuning

  

  

  Haotian Liu et al., University of Wisconsin-Madison and Microsoft. A vision encoder, a linear projection, an open LLM, and instruction data generated by GPT-4. An open GPT-4V-style model anyone could train.

• 2023.07: RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

  

  

  Anthony Brohan et al., Google DeepMind. A web-pretrained vision-language model fine-tuned to emit robot actions as text tokens. Knowledge from the web transfers to manipulation.

• 2023.09: GPT-4V System Card

  

  

  OpenAI. Vision added to a frontier model; the document is about capabilities and risks rather than architecture. Multimodality becomes a product.

• 2023.10: Open X-Embodiment: Robotic Learning Datasets and RT-X Models

  

  

  21 institutions led by Google DeepMind. One million robot trajectories from 22 robot types pooled into one dataset; RT-X models trained on it generalize across embodiments. Robotics gets its shared corpus.

• 2024.03: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

  

  

  Google DeepMind. A mixture-of-experts model with a context of millions of tokens: whole books, hours of video, entire code bases in one prompt. Long context becomes practical.

• 2024.05: Octo: An Open-Source Generalist Robot Policy

  

  

  Octo Model Team, UC Berkeley and others. An open generalist robot policy, a 93M-parameter transformer trained on 800k trajectories, fine-tunable to a new robot in hours. The foundation-model idea arrives for robots.

• 2024.06: OpenVLA: An Open-Source Vision-Language-Action Model

  

  

  Moo Jin Kim et al., Stanford, Berkeley, Google DeepMind, Toyota Research Institute. An open 7B vision-language-action model trained on 970k Open X-Embodiment episodes, beating RT-2-X with 7× fewer parameters. Open weights for robot control.

• 2024.08: GPT-4o System Card

  

  

  OpenAI. One model trained end to end on text, audio and images, with real-time speech. Native multimodality replaces pipelines of separate models.

• 2025.01: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

  

  

  DeepSeek. Reinforcement learning with verifiable rewards, a correct answer or passing tests, applied directly to a language model; long chain-of-thought reasoning emerges on its own. Open weights at the level of OpenAI o1; RL returns to the center of LLM training.