Vladimir_Nesov comments on The Urgent Meta-Ethics of Friendly Artificial Intelligence - Less Wrong

45 Post author: lukeprog 01 February 2011 02:15PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (249)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 02 February 2011 01:12:34AM *  3 points [-]

Logical uncertainty, which is unavoidable no matter how smart you are, blurs the line. AI won't "understand" expected utility maximization completely either, it won't see all the implications no matter how much computational resources it has. And so it needs more heuristics to guide its decisions where it can't figure out all the implications. Those are the counterparts of deontological injunctions, although of course they must be subject to revision on sufficient reflection (and what "sufficient" means is one of these injunctions, also subject to revision). Some of then will even have normative implications, in fact that's once reason preference is not utility function.