Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Vladimir_Nesov comments on Eutopia is Scary - Less Wrong

33 Post author: Eliezer_Yudkowsky 12 January 2009 05:28AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (121)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Vladimir_Nesov 12 January 2009 09:04:19PM 2 points [-]

Allowing public dissemination of science is analogous to allowing overly fast increase in intelligence, in several respects.

First, in both cases you get some answers without enjoying the process of obtaining them. To solve a problem, you may learn an answer in a textbook, or find it using raw power of increased intelligence. Problems which you might've enjoyed are now spoiled, with answer either known or boringly trivial.

Second, greater knowledge or intelligence opens the ability to enjoy richer problems, so it might be a good idea to move from chimp-level problems to human-level problems even if it spoils the fun of solving chimp-level problems as a chimp. It might be a good idea to start from deeper background in science and better intelligence, preferably synchronized with each other, if still below the level where the difference becomes unclear.

And third, you are allowed or even encouraged to hurt yourself. If the path you choose as a result of being guided by external knowledge or intelligence leads in the wrong direction, that influence needs to be controlled. It looks like the optimal path should consist of a combination of local development by one's own power, and influence of superintelligent overseer that prevents heading in the wrong direction in the long term.