This is a linkpost for https://foundational-research.org/s-risks-talk-eag-boston-2017/
I didn't realize then that disutility of human-built AI can be much larger than utility of FAI, because pain is easier to achieve than human utility (which doesn't reduce to pleasure). That makes the argument much stronger.