DanArmak comments on Holden's Objection 1: Friendliness is dangerous - LessWrong

11 Post author: PhilGoetz 18 May 2012 12:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (428)

You are viewing a single comment's thread. Show more comments above.

Comment author: JoshuaZ 18 May 2012 01:38:26AM *  6 points [-]

Opponents of gay marriage make it clear that they do not object to gay marriage based on a long-range utilitarian calculation; they directly value not allowing gays to marry.

Well, most of them do so in part out of their deity telling them that that's a value. If the extrapolated CEV takes into account that they are just wrong about there being such a deity, it should respond accordingly. (I'm working under the what should not be controversial assumption that the AGI isn't going to find out that in fact there is such a deity hanging around.)

Comment author: DanArmak 19 May 2012 03:57:29PM 1 point [-]

Extrapolated CEV would be working from observable evidence + a good prior. Whereas lots of people insist it's very important to them to believe in a deity through faith, despite any contrary evidence (let alone lack of evidence). How are you going to tell the CEV to ignore such values?