rwallace comments on Hedging our Bets: The Case for Pursuing Whole Brain Emulation to Safeguard Humanity's Future - Less Wrong

11 Post author: inklesspen 01 March 2010 02:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (244)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 01 March 2010 04:57:30AM 7 points [-]

It would have been good to check this suggested post topic on an Open Thread, first - in fact I should get around to editing the FAQ to suggest this for first posts.

Perhaps the AI rules wisely and well, and can give us anything we want, "save relevance".

In addition to the retreads that others have pointed out on the upload safety issue, this is a retread of the Fun Theory Sequence:

http://lesswrong.com/lw/xy/the_fun_theory_sequence/

Also the way you phrased the above suggests that we build some kind of AI and then discover what we've built. The space of mind designs is very large. If we know what we're doing, we reach in and get whatever we specify, including an AI that need not steal our relevance (see Fun Theory above). If whoever first reaches in and pulls out a self-improving AI doesn't know what they're doing, we all die. That is why SIAI and FHI agree on at least wistfully wishing that uploads would come first, not the relevance thing. This part hasn't really been organized into proper sequences on Less Wrong, but see Fake Fake Utility Functions, the Metaethics sequence, and the ai / fai tags.

Comment author: rwallace 03 March 2010 02:30:31AM 1 point [-]

That is why SIAI and FHI agree on at least wistfully wishing that uploads would come first

It seems to me that uploads first is quite possible, and also that the relatively small resources currently being devoted to uploading research, make the timing of uploading, a point of quite high leverage. Would SIAI or FHI be interested in discussing ways to accelerate uploading?

Comment author: CarlShulman 03 March 2010 02:35:08AM 2 points [-]
Comment author: rwallace 03 March 2010 05:08:48AM 0 points [-]

Thanks! Excellent start. The section on falsifiable design, in particular, I'd recommend reading for anyone interested in any kind of speculative technology.