Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Nick_Tarleton comments on Can't Unbirth a Child - Less Wrong

24 Post author: Eliezer_Yudkowsky 28 December 2008 05:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (94)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Nick_Tarleton 29 December 2008 03:14:44AM 0 points [-]

Anyways, why not build an AI who's goal was to non-coercively increase the intelligence of mankind? You don't have to worry about its utility function being compatible with ours in that case. Sure I don't know how we'd go about making human intelligence more easily modified (as I have no idea what sentience is), but a super-intelligence might be able to figure it out.

And it doesn't consider it significant that this one hack that boosts IQ by 100 points makes us miserable/vegetables/sadists/schizophrenic/take your pick. Or think that it should have asked before turning the rest of the solar system into computronium. And, of course, it won't hold with the existence of anything intelligent enough to potentially turn it off, and so on....

The Hidden Complexity of Wishes