Jeffrey_Soreff comments on Value is Fragile - Less Wrong

41 Post author: Eliezer_Yudkowsky 29 January 2009 08:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (88)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Jeffrey_Soreff 30 January 2009 12:00:33AM 0 points [-]

@Jordan - agreed.

I think the big difference in expected complexity is between sampling the space of possible singletons' algorithms results and sampling the space of competitive entities. I agree with Eliezer that an imprecisely chosen value function, if relentlessly optimized, is likely to yield a dull universe. To my mind the key is that the ability to relentlessly optimize one function only exists if a singleton gets and keeps an overwhelming advantage over everything else. If this does not happen, we get competing entities with the computationally difficult problem of outsmarting each other. Under this scenario, while I might not like the detailed results, I'd expect them to be complex to much the same extent and for much the same reasons as living organisms are complex.