You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

chaosmage comments on Morality Isn't Logical - Less Wrong Discussion

19 Post author: Wei_Dai 26 December 2012 11:08PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (85)

You are viewing a single comment's thread.

Comment author: chaosmage 27 December 2012 01:03:56AM 0 points [-]

You're mischaracterizing the quote that your post replies to. EY claims that he is attempting to comprehend morality as a logical, not a physical thing, and he's trying to convince readers to do the same. You're evidently thinking of morality as a physical thing, something essentially derived from the observation of brains. You're restating the position his post responds to, without strengthening it.

Comment author: timtyler 29 December 2012 12:51:18PM *  -2 points [-]

The argument in that post seems incoherent to me. In the conventional natural sciences, what is moral is the subject matter of biology. This employs game theory and evolutionary theory (i.e. logic), but also considers the laws of physics and the local state of the universe to explain existing moral systems.

For instance, consider the question of whether it is wrong to drive on the left-hand side of the road. That isn't logic, it depends on the local state of the universe. Two advanced superintelligences which had evolved independently could easily find themselves in disagreement over such issues.

This is an example of spontaneous symmetry breaking. It is one of the factors which explains how arbitrarily-advanced agents can still disagree on what the right thing to do is.