Wei_Dai comments on Why No Wireheading? - Less Wrong

16 [deleted] 18 June 2011 11:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (112)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 22 June 2011 10:22:36PM 3 points [-]

What's the evolutionary advantage of an agent that has highly volatile terminal values and can easily be hijacked,

Humans seem to act as general meme hosts. It seems fairly easy for a human to be hijacked by a meme in a way that decreases their genetic inclusive fitness. Presumably this kind of design at least had an evolutionary advantage, in our EEA, or we wouldn't be this way.

or relies on fairly advanced circuitry to even do value calculations?

If you can host arbitrary memes, then "external referent consequentialism" doesn't really need any extra circuitry. You just have to be convinced that it's something you ought to do.