cousin_it comments on The Gift I Give Tomorrow - Less Wrong

28 Post author: Raemon 11 January 2012 04:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (33)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheOtherDave 11 January 2012 04:49:08PM 2 points [-]

I completely agree about the ability to circumvent humanity's objections, either by propaganda as you describe, or just by ignoring those objections altogether and doing what it thinks best. Of course, if for whatever reason the system were designed to require uncoerced consent before implementing its plans, it might not use that ability. (Designing it to require consent but to also be free to coerce that consent via superstimuli seems simply silly: neither safe nor efficient.)

Comment author: cousin_it 11 January 2012 05:25:25PM 6 points [-]

This thread has made it clear to me that designing a superintelligence that asks for consent before doing stuff is a bad idea. Thanks!