LucasSloan comments on Harry Potter and the Methods of Rationality discussion thread, part 2 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (696)
A person is reflectively self aware.
Evolution built me to care about humans, and upon reflection, the values I have include non-humans who have features like being reflectively self aware.
Is that what you would want to want, given the option, or is that a lizard-brain instinct that gets in the way of your ability to evaluate what's really the right thing to do?
I can still interpret that either way. Do you mean that on reflection you realize that you emotionally desire that, or that on reflection you *decide" that that's what's important?
There's also Hayekian arguments-- self-aware agents are apt to accumulate information about their own desires and activities. Systems which allow that information to have an effect seem to be more capable.