Eliezer_Yudkowsky comments on Conjuring An Evolution To Serve You - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (22)
Put more simply, in artificial evolution you get exactly what the fitness function you've written asks for
You don't even necessarily get that. The animal breeders thought they were asking for more eggs. They did get some eggs, but with side effects, and not nearly as many eggs as they could have gotten, if they'd used a different breeding format with the same fitness function: fitness=eggs, vehicle=group instead of fitness=eggs, vehicle=individual.
They could well have gotten fewer eggs by breeding for eggs, as Enron did, if the chickens had discovered enough negative-sum tricks, as Enron did.