Vladimir_Nesov comments on Open Thread June 2010, Part 3 - Less Wrong

6 Post author: Kevin 14 June 2010 06:14AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (606)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 18 June 2010 05:07:02PM *  2 points [-]

But if the first difference is much bigger than the second, the pattern is the reverse.

It's not literally the reverse, because if you don't create those FAIs, nobody will, and so the UFAIs won't have the incentive to give you your small share. It's never good to increase probability of UFAI at the expense of probability of FAI. I'm not sure whether there is any policy guideline suggested by these considerations, conditional on the pattern in utility you discuss. What should we do differently depending on how much we value small vs. large control? It's still clearly preferable to have UFAI to having no future AI, and to have FAI to having UFAI, in both cases.

Comment author: CarlShulman 19 June 2010 02:23:44AM 0 points [-]

Worrying less about our individual (or national) shares, and being more cooperative with other humans or uploads seems like an important upshot.