Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Richard_Hollerith2 comments on Mirrors and Paintings - Less Wrong

12 Post author: Eliezer_Yudkowsky 23 August 2008 12:29AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (41)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Richard_Hollerith2 24 August 2008 02:29:24AM -2 points [-]

(I have removed references to the notion of the moral miracle from my blog entry because I have not had time to make sure I understand what you mean by the notion.)

Eliezer, in this blog entry and in others, you appeal to human qualities like love, laughter and humor to support a claim that a sufficiently powerful intelligence can learn all it needs to know about rightness or about proper terminal values by examining the human brain. If the appeals do not support your claim then please explain why they occur often in your posts about your claim.

I humbly submit that that is drawing the target where the arrow landed because the criteria (love, laughter, etc) you are using to score the goodness of the human brain (as a sort of reference book about terminal values) are an effect of the human brain.

If another species (a dinosaur, say) had evolved to our stage and could understand your blog entries, there is a good chance that it would be quite unmoved by your appeals to love, laughter, etc. In other words, if the arrow landed over there (where the tool-using mental-model-making dinosaur species is) then the causal chain that would have led a mind to draw the target where you drew the target would not exist.

Heck, you explicitly made the point about a week ago that you would not have expected the first tool-using mental-model-making species to have a sense of humor. In other words, if the arrow had landed somewhere else, there would exist no mind able to comprehend humor, so the target would not have been drawn around the points in the space of possible minds that have a sense of humor.

In summary, that one sentence by Caledonian is a sound and relevant criticism (which should not be taken as an endorsement of his other comments).