Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Richard_Hollerith2 comments on Complex Novelty - Less Wrong

26 Post author: Eliezer_Yudkowsky 20 December 2008 12:31AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (65)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Richard_Hollerith2 22 December 2008 08:10:28PM 0 points [-]

Modifying yourself that way would just demonstrate that you value the means of fun more than the ends. Even if you could make that modification, would you?

Yes, Ben Jones, I sincerely would. (I also value the means of friendship, love, sex, pleasure, health, wealth, security, justice, fairness, my survival and the survival of my friends and loved ones more than the ends. I have a very compact system of terminal values. I.e., very few ultimate ends.)

I am fully aware that my saying that I value friendship as a means to an end rather than an end in itself handicaps me in the eyes of prospective friends. Ditto love and prospective lovers. But I am not here to make friends or find a lover.

People have a bias for people with many terminal values. Take for example a person who refuses to eat meat because doing so would participate in the exploitation of farm animals. My hypothesis is that that position helps the person win friends and lovers because prospective friends and lovers think that if the person is that scrupulous towards a chicken he has never met then he is more likely than the average person to treat his human friends scrupulously and non-exploitatively. A person with many terminal values is trusted more than a person with with fewer and is rarely called on to explain the contradictions in his system of terminal values.

There are commercials for cars in which the employees of the car company are portrayed as holding reliable cars with zero defects as a terminal value. Or great-tasting beer as a terminal value. And of course advertiser tend to keep using a pitch only if it helps sell more cars or beer. It is my hope that some of the readers of these words realize that there is something wrong with an agent of general intelligence (a human in this case or an organization composed of humans) holding great-tasting beer as a terminal value.

I invite the reader to believe with me that Occam's Razor -- that everything else being equal, a simple system of beliefs is to be preferred over a complex system -- applies to normative beliefs as well as positive beliefs. Moreover, since there is nothing that counts as evidence for or against a normative belief, a system of normative beliefs should not grow in complexity as the agent gathers evidence from its environment the way a system of positive beliefs does.

Finally, if Vladimir Slepnev has written up his ethical beliefs, I ask him to send them to me.