Wei_Dai comments on Resetting Gandhi-Einstein - Less Wrong

9 Post author: Stuart_Armstrong 13 June 2011 10:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (32)

You are viewing a single comment's thread.

Comment author: Wei_Dai 14 June 2011 11:41:20PM *  7 points [-]

Carl Shulman wrote about resetting uploads to prevent value change in his Whole Brain Emulation and the Evolution of Superorganisms (which I previously posted under discussion):

The methods outlined above to enhance productivity could also be used to produce emulations with trusted motivations. A saved version of an emulation would have particular motives, loyalties, and dispositions which would be initially shared by any copies made from it. Such copies could be subjected to exhaustive psychological testing, staged situations, and direct observation of their emulation software to form clear pictures of their loyalties. Ordinarily, one might fear that copies of emulations would subsequently change their values in response to differing experiences (Hanson and Hughes, 2007). But members of a superorganism could consent to deletion after a limited time to preempt any such value divergence. Any number of copies with stable identical motivations could thus be produced, and could coordinate to solve collective action problems even in the absence of overarching legal constraints.

Comment author: Stuart_Armstrong 15 June 2011 10:05:59PM 2 points [-]

No new ideas under the sun... :-)