Eliezer_Yudkowsky comments on Conjuring An Evolution To Serve You - Less Wrong

39 Post author: Eliezer_Yudkowsky 19 November 2007 05:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (22)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 19 November 2007 10:56:23PM 4 points [-]

Put more simply, in artificial evolution you get exactly what the fitness function you've written asks for

You don't even necessarily get that. The animal breeders thought they were asking for more eggs. They did get some eggs, but with side effects, and not nearly as many eggs as they could have gotten, if they'd used a different breeding format with the same fitness function: fitness=eggs, vehicle=group instead of fitness=eggs, vehicle=individual.

They could well have gotten fewer eggs by breeding for eggs, as Enron did, if the chickens had discovered enough negative-sum tricks, as Enron did.