= 27d567bc2d48d9f40e9ccc20ea370b15)
army1987 comments on No Universally Compelling Arguments in Math or Science - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (227)
Clippy knows what is moral and what isn't. He just doesn't care.
Imagine if humans had never broken into different groups and we all spoke the same language. No French, no English, just "the Language". People study the Language, debate it, etc.
Then one day intelligent aliens arrive. Philosophers immediately begin debating: do these aliens have the Language? One the one hand, they're making noises with what appears to be something comparable to a mouth, the noises have an order and structure to them, and they communicate information. But what they do sounds nothing like "the Language". They refer to objects with different sounds than the Language requires, and sometimes make sounds that describe what an object is like after the sound that refers to the object.
"Morality" has a similar type-token ambiguity. It can refer to our values or to values in general. Saying Clippy knows what is moral but that he doesn't care is true under the token interpretation, but not the type one. The word "morality" has meanings and connotations that imply that Clippy has a morality but that it is just different-- in the same way that the aliens have language but that it is just different.
So, I guess the point of EY's metaethics can be summarized as ‘by “morality” I mean the token, not the type’.
(Which is not a problem IMO, as there are unambiguous words for the type, e.g. “values” -- except insofar as people are likely to misunderstand him.)
Especially because the whole point is to optimize for something. You can't optimize for a type that could have any value.
How is this different from:
The creationist knows what I believe but doesn't care.
The argument of the dragon in my garage suggests that the supernaturalist already knows the facts of the natural world, but doesn't care.
But the sense in which "Clippy knows what is moral" is that Clippy can correctly predict humans, and "morality" has to do with what humans value and approve of — not what paperclippers value and approve of.
A creationist is mistaken about the origin of the Earth (they believe the Earth was created by a deity).
Aumann's agreement theorem prevents that from happening to ideal epistemic rationalists; there's no analogue for instrumental rationality.
But...
Aumann's agreement theorem assumes common priors, what I described can only happen to instrumental rationalists with different utility functions. So the question is why we expect all rationalists to use One True Prior (e.g. Solomonoff induction) but each to use their own utility function.