David Krueger

I don't want to summarize too much, since I think everyone should read this latest piece by Yoshua Bengio (Turing award winner for Deep Learning, along with Geoff Hinton and Yann Lecun): Suffice to say that this piece engages w/AI x-risk very seriously.

BTW, 2 papers I often recommend: 1) "The alignment problem from a deep learning perspective" @RichardMCNgo et al. for research overview: 2) "Natural Selection Favors AIs over Humans" @DanHendrycks for argument for risk:

BTW, 2 papers I often recommend: 1) "The alignment problem from a deep learning perspective" @RichardMCNgo et al. for research overview: 2) "Natural Selection Favors AIs over Humans" @DanHendrycks for argument for risk:

This is a good point / thread. Note that @MIRIBerkeley and @ESYudkowsky deserve credit for (purportedly) focusing on inner alignment from the beginning. The first public documentation of this I'm aware of are Alignment Forum posts from 2017, e.g.:

Great piece from @_achan96_ about why "solving alignment" is not enough to guarantee good outcomes. IME, this is a common objection from ML researchers who are concerned about risks from advanced AI, but skeptical of AI Alignment.

Recommended! This is a well-written, concise (10 pages!) summary of the Alignment Problem. It helps explain: 1) The basic concerns of (many) alignment researchers 2) Why we don't think it's crazy to talk about AI killing everyone 3) What's distinctive about the field.

This thread by @jachiam0 makes a lot of good points that seem underappreciated by many in AI x-safety. But I have several significant disagreements I want to highlight:

Another post by @jenner_erik and Johannes Treutlein presents a great response: Reflecting, I think Katja's post mostly gives me optimism about slow take-off; a lot of my pessimism there stems from general pessimism about advanced info-tech + society.