khafra comments on Mahatma Armstrong: CEVed to death. - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (60)
Presumably this means "all the morally relevant facts," since giving State 1 "all the facts" would be isomorphic to presenting him with the argument-simulation. But determining all the morally relevant facts is a big part of the problem statement. If the AI could determine which aspects of which actions were morally relevant, and to what the degree and sign of that moral valence was, it wouldn't need CEV.
We could lock down the argument more, just to be safe.
I'm not sure whether a text-only channel between State 1 and State 7, allowing only if-then type statements with a probability attached, would allow brainwashing or hypnosis. But I'm also not sure how many State 1 racists would be convinced that racism is unethical, over such a channel.
How about the individual versions at State 1 and State 7 both get all the facts that they consider relevant themselves? And maybe a State 1 racist really wouldn't have CEV towards non-racism- we just have to accept that.