Palus Astra

Just a year ago we released a two part episode titled An Overview of Technical AI Alignment with Rohin Shah. That conversation provided details on the views of central AI alignment research organizations and many of the ongoing research efforts for designing safe and aligned systems. Much has happened in the past twelve months, so we've invited Rohin — along with fellow researcher Buck Shlegeris — back for a follow-up conversation. Today's episode focuses especially on the state of current research efforts for beneficial AI, as well as Buck's and Rohin's thoughts about the varying approaches and the difficulties we still face. This podcast thus serves as a non-exhaustive overview of how the field of AI alignment has updated and how thinking is progressing. Topics discussed in this episode include: * Rohin's and Buck's optimisms and pessimism about different approaches to aligned AI * Traditional arguments for AI as an x-risk * Modeling agents as expected utility maximizers * Ambitious value learning and specification learning/narrow value learning * Agency and optimization * Robustness * Scaling to superhuman abilities * Universality * Impact regularization * Causal models, oracles, and decision theory * Discontinuous and continuous takeoff scenarios * Probability of AI-induced existential risk * Timelines for AGI * Information hazards You can find the page for this podcast here: https://futureoflife.org/2020/04/15/an-overview-of-technical-ai-alignment-in-2018-and-2019-with-buck-shlegeris-and-rohin-shah/ Transcript Note: The following transcript has been edited for style and clarity. Lucas Perry: Welcome to the AI Alignment Podcast. I’m Lucas Perry. Today we have a special episode with Buck Shlegeris and Rohin Shah that serves as a review of progress in technical AI alignment over 2018 and 2019. This episode serves as an awesome birds eye view of the varying focus areas of technical AI alignment research and also helps to develop a sense of the