Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Zubon comments on Eutopia is Scary - Less Wrong

33 Post author: Eliezer_Yudkowsky 12 January 2009 05:28AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (121)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Zubon 12 January 2009 01:50:39PM 13 points [-]

I should toss out my own. The first that occurs to me is that utopia could be far more constrained rather than far freer. Most of us seem to have a vision of a universe where you can do whatever you want so long as it harms no one else, a kind of Nozickian utopia of utopias. We will all move to Permutation City or build Prime Intellect, and then your only restraint is that you cannot restrain others.

If freedom is only instrumentally useful, rather than morally fundamental, there is no reason for this given sufficient powers of prediction and control. If Asimov's supercomputers really can get around the Hayekian knowledge problem and perfect the economy, most arguments against command economies just went out the window. If I really do know better than you, with no epistemic issues, giving you more freedom is like letting a child play in traffic. If I can prove to you with mathematical certainty that decision X would make things worse, and you still choose X (objectively and by your preferences), we have just proved that you are not capable of handling freedom.

Telling someone where the mines are takes all the fun out of Minesweeper, but you should do it IRL if the town is on the edge of an old war zone. If someone has the walkthrough for my life, I may not consult it constantly, but I would like a pop-up confirmation window to appear whenever a decision will lead to "game over." I would also like more save points.

Comment author: Aegeus 09 February 2011 03:34:49AM 2 points [-]

That actually seems like a good idea. Choice paralysis seems like it would be a serious problem for any "Do anything you can imagine" utopia, because I can't think of what I'd do if I had the power to do anything.

Or, put another way, Minecraft became a lot more fun once it took away the infinite supply of blocks.

Comment author: DSimon 07 April 2011 08:29:19PM 0 points [-]

Choice paralysis seems like it would be a serious problem for any "Do anything you can imagine" utopia, because I can't think of what I'd do if I had the power to do anything.

How about "myself twice as smart, check to see if I still have don't have a really good idea what to do, do it if so, if not repeat the process"?