They agree you can’t control superintelligence. So why do they disagree about whether to build it? This episode is brought to you by AgelessRx, where aging becomes a puzzle that can be solved. The mission is to help you live longer and healthier by adding more life to your years through science-backed solutions and personalized care. Learn more at http://agelessrx.com/beyond-tomorrow. Two people with opposite P(doom) numbers go head to head. Roman Yampolskiy puts his close to 100%, and Mike Israetel puts his near zero. The host reads out a series of big statements, and the two take them apart one at a time. Mike argues that a smarter-than-human system will be a better cooperator than we are: it will want to study us and keep us around, and it will out-moralize us. Roman argues that nobody knows how to encode that, that a superintelligence won’t need us, and that you can’t control something far smarter than you. A special guest joins midway to argue that a moral AGI is an engineering problem we can solve, and that the pause debate is a distraction. The conversation then moves through whether a pause is possible, whether many superintelligences can police each other, what to do about bioweapons, and whether either of them would upload into the cloud. It ends with where each thinks we’ll be in October 2027, plus a lightning round on protein, creatine, and AI myths. Key Takeaways Mike: superintelligence will treat us as valuable allies, so P(doom) falls as intelligence rises. Roman: without encoded care, a system that no longer needs us has no reason to keep us Both agree superintelligence can’t be permanently controlled. Mike says don’t try. Roman says don’t build it Meet the Guests Roman Yampolskiy is an AI safety researcher and computer scientist known for his work on the limits of AI control. He argues superintelligence can’t be made safe, and wants the field to focus on narrow, useful tools instead. Mike Israetel is a sports scientist and fitness educator, and one of the more optimistic voices in the AI debate. He expects an age of transformation and prosperity, with a P(doom) he describes as roughly zero. Ben Goertzel is the CEO of SingularityNET and Executive Chairman & Chief Scientist of BGI Labs who popularized the term “artificial general intelligence.” He’s building open-source, decentralized AGI. ⏱ Timestamps 00:00 Intro 01:38 Opening Positions and P(Doom) 04:51 Caring or Murderous Superintelligence? 11:00 Mosquitoes, Factory Farms, and ASI 23:04 What Can Humans Contribute? 29:26 Brain Uploads and Survival 35:56 Can We Build Moral AGI? 37:33 Is Morality a Science? 40:36 Negative Utilitarianism Explained 52:09 Superintelligence in 10 Years? 53:56 Should We Pause AI? 1:02:49 Hardware Chip Pause Idea 1:04:55 The Viral Resignation Tweet 1:10:20 Narrow AI vs AGI 1:11:27 AI Solving Math Problems 1:20:23 How Much AI Is Enough? 1:23:35 Can Rival ASIs Police Each Other? 1:29:36 Can We Control Superintelligence? 1:34:18 Would You Build ASI? 1:39:45 Alignment Without Equality 1:46:16 One ASI or Many? 1:49:52 Who Controls Superintelligence? 1:52:27 Does Intelligence Make You Benevolent? 1:57:04 Would You Upload Your Brain? 2:01:42 Should Musk Set AI Rules? 2:03:02 Is AI Conscious? 2:05:23 AI Bioweapon Risk 2:07:26 Longevity and Living to 150 2:11:26 AI in October 2027 2:14:22 AI and Developing Countries 2:16:01 Final Message to the World Related Episodes With Mike: https://youtu.be/mdiRf_SNFxw?si=fpUQ6V4YGb37sqpj With Roman: https://youtu.be/QkpOFlj604s?si=W4pL-jyu1_WtbHcq With Ben: https://www.youtube.com/watch?v=RS4LCAa5NQ4 Connect with Roman: Website: https://www.romanyampolskiy.com/ Facebook: https://www.facebook.com/roman.yampolskiy/ LinkedIn: https://www.linkedin.com/in/romanyam/ X: https://x.com/romanyam YouTube: https://www.youtube.com/@RomanYampolskiy Connect with Mike: Website: rpstrength.com/ Facebook: facebook.com/michael.israetel Instagram: @drmikeisraetel X: @misraetel YouTube: https://www.youtube.com/@MikeIsraetelMakingProgress Connect with Ben Website: https://singularitynet.io/ LinkedIn: https://www.linkedin.com/in/bengoertzel X: https://x.com/bengoertzel Connect with Julian Instagram: @thebeyondtomorrowpodcast / @juliankissa LinkedIn: julian-issa X: @juliankissa Website: www.beyondtomorrowpodcast.com Email: bookings@beyondtomorrowpodcast.com
Please sign in to join the discussion
Sign InNo comments yet. Be the first to share your thoughts!