Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Bogdan_Butnaru2 comments on Failed Utopia #4-2 - Less Wrong

52 Post author: Eliezer_Yudkowsky 21 January 2009 11:04AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (248)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Bogdan_Butnaru2 21 January 2009 01:51:32PM 6 points [-]

I was just thinking: A quite perverse effect in the story would be if the genie actually _could_ have been stopped and/or improved: That is, its programming allowed it to be reprogrammed (and stop being evil, presumably leading to better results), but due to the (possibly complex) interaction between its 107 rules it didn't actually have any motivation to reveal that (or teach the necessary theory to someone) before 90% of people decided to kill it.