MixedNuts comments on Three Worlds Decide (5/8) - Less Wrong

24 Post author: Eliezer_Yudkowsky 03 February 2009 09:14AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (136)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: MixedNuts 28 June 2011 07:27:52PM 3 points [-]

Does not follow. We value diversity, but not diversity of utility functions. This allows diversity because some terms in the utility function are subjective (i.e. they depend on the agent's brain). If I have "people should drink tasty beverages" as a terminal value, I have "MixedNuts should drink mango juice" and "Helen should drink tea", and if a cosmic ray alters Helen's brain, the latter will change. Helen.tasty is not an approximation of a mysterious Platonic essence of absolute tastiness, unlike Helen.prove_theorem.

(Maybe we also want some diversity in terminal values, but it'll probably be very small - we aren't going to sacrifice the galaxy to cheesecake makers.)