Biography:Geoffrey Irving
Geoffrey Irving is a computer scientist and AI safety researcher. His work includes research on scalable oversight, AI safety via debate, and early applications of reinforcement learning from human feedback (RLHF) to large language models.[1][2] He has worked at OpenAI, Google DeepMind, and the United Kingdom's AI Safety Institute, and has been featured on the TIME100 AI list.[3]
Education and career
Irving received a PhD in computer science from Stanford University in 2007. His dissertation was titled Methods for the Physically Based Simulation of Solids and Fluids.[4]
He has worked at Google Brain, OpenAI, and Google DeepMind.[3] At DeepMind, he led the Scalable Alignment Team.[5] In 2024, he joined the United Kingdom's AI Safety Institute, where he served as a research director and later as chief scientist.[6][3]
In 2026, Irving became a co-founder and chief scientist of Sequent Research, an artificial intelligence alignment research organization.[7]
Research
Irving's work on scalable oversight includes AI Safety via Debate, a 2018 paper with Paul Christiano and Dario Amodei. The paper proposed using debates between AI systems to help a human judge evaluate answers to difficult questions.[8] The debate approach has been discussed as a method for improving the honesty and oversight of advanced AI systems.[1]
Irving was also a coauthor of Fine-Tuning Language Models from Human Preferences, a 2019 OpenAI paper that applied human-preference reward modeling to pretrained language models.[2] The paper was an early contribution to reinforcement learning from human feedback (RLHF) for large language models.
With philosopher Amanda Askell, Irving wrote a 2019 article in Distill arguing that AI safety research should make greater use of empirical social science.[9]
References
- ↑ 1.0 1.1 Ornes, Stephen (8 November 2024). "Debate May Help AI Models Converge on Truth". Quanta Magazine. https://www.quantamagazine.org/debate-may-help-ai-models-converge-on-truth-20241108/. Retrieved 24 June 2026.
- ↑ 2.0 2.1 Ziegler, Daniel M.; Stiennon, Nisan; Wu, Jeffrey; Brown, Tom B.; Radford, Alec; Amodei, Dario; Christiano, Paul; Irving, Geoffrey (2019). "Fine-Tuning Language Models from Human Preferences". arXiv:1909.08593 [cs.CL].
- ↑ 3.0 3.1 3.2 Perrigo, Billy (5 September 2024). "Geoffrey Irving". Time. https://time.com/collections/time100-ai-2024/7012887/geoffrey-irving/. Retrieved 24 June 2026.
- ↑ Irving, Geoffrey (2007). Methods for the Physically Based Simulation of Solids and Fluids (PhD thesis). Stanford University. Retrieved 24 June 2026.
- ↑ "AI Safety Institute: third progress report". 5 February 2024. https://www.gov.uk/government/publications/uk-ai-safety-institute-third-progress-report/ai-safety-institute-third-progress-report.
- ↑ Hornstein, Oscar (5 February 2024). "AI Safety Institute recruits Google DeepMind researcher". UKTN. https://www.uktech.news/ai/ai-safety-institute-research-director-20240205.
- ↑ Irving, Geoffrey; Holness-Tofts, Alex; Hoogland, Jesse; Murfet, Daniel; Pfau, Jacob; Cozzi, Marco; van Wingerden, Stan (10 June 2026). "Sequent: scale and automation for higher confidence in alignment". https://www.alignmentforum.org/posts/AP7YDke5jjY4v3X9Z/sequent-scale-and-automation-for-higher-confidence-in-1.
- ↑ Irving, Geoffrey; Christiano, Paul; Amodei, Dario (2018). "AI Safety via Debate". arXiv:1805.00899 [stat.ML].
- ↑ Irving, Geoffrey; Askell, Amanda (19 February 2019). "AI Safety Needs Social Scientists". Distill 4 (2). doi:10.23915/distill.00014. https://distill.pub/2019/safety-needs-social-scientists/.
External links
 |  |
