Psychohistorian comments on You cannot be mistaken about (not) wanting to wirehead - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (79)
Let me try a different tack here. Suppose you have in front of you two flavors of ice cream. You don't know what they taste like, but you prefer the red one because you like red and that's the only thing you have to go on. Now an FAI comes along and tells you that it predicts if you knew what the flavors taste like, you'd choose the blue one instead. Do you not switch to the blue one?
Know that it's the "best" is hardly having full declarative knowledge, when we don't know how good "best" is.
I don't see how that makes any sense, given my ice cream example.
There's a rather enormous leap between the FAI saying, "Y'know, I think you'd like that one more," and the FAI altering your brain so you select that one. Providing new information simply isn't altering someone's mind in this context.