Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

[link] Whole Brain Emulation and the Evolution of Superorganisms

17 Post author: Wei_Dai 03 May 2011 11:38PM

Robin Hanson has made several recent posts on Overcoming Bias about upload economics. I remain mystified why he doesn't link to or otherwise reference or comment on Carl Shulman's 2010 paper, Whole Brain Emulation and the Evolution of Superorganisms, which mentions many of the same ideas and seems to have taken them to their logical conclusions. I was going to complain again in the comments section over there, but then I noticed that the paper hasn't been posted or discussed here either. So here's the abstract. (See above link for the full paper.)

Many scientists expect the eventual development of intelligent software programs capable of closely emulating human brains, to the point of substituting for human labor in almost every economic niche. As software, such emulations could be cheaply copied, with copies subsequently diverging and interacting with their copy-relatives. This paper examines a set of evolutionary pressures on interaction between related emulations, pressures favoring the emergence of superorganisms, groups of emulations ready to self-sacrifice in service of the superorganism. We argue that the increased capacities and internal coordination of such superorganisms could pose increased risks of overriding human values, but also could facilitate the solution of global coordination problems.

Comments (7)

Comment author: Wei_Dai 16 June 2011 05:38:40PM 11 points [-]

I think this paper offers an object lesson in taking ideas seriously. I mean I thought I had taken the idea of uploads seriously, but apparently not and neither had anyone else. The conclusions reached in the paper do not seem to require really deep insight or information that is difficult to gather or process. Instead they seem like the obvious conclusions that anyone would reach if they just thought seriously about what would really happen once uploads became possible.

(Of course "obvious conclusions that anyone would reach if ..." is an exaggeration, but I think you get my point. :)

Comment author: Kaj_Sotala 04 May 2011 12:27:21PM 4 points [-]

This is an excellent (although scary) paper, much recommended.

Comment author: RobinHanson 05 October 2011 12:04:39PM 3 points [-]
Comment author: Vladimir_Nesov 05 October 2011 05:25:25PM 3 points [-]

(For the reference: these comments are discussed on LW here.)

Comment author: FiftyTwo 28 May 2012 03:31:14PM 1 point [-]

That was very helpful. Thanks for thinking of people of the future.

Comment author: DanielVarga 04 May 2011 12:13:29PM *  4 points [-]

Robin has many-many clever and insightful observations, but I think the history books will only remember him for these unbelievably naive em posts. People of the future will find his idea of an em just as charmingly ridiculous as we see flapping-wing aeroplanes, and they will often quote him for comedic effect. At least that's my hope, because I think he would deserve that for the hubris with which he builds these complex intellectual constructions on quicksand.

More seriously, this whole em series is another sterile example of what I called Individualism Bias at this old thread. Johnicolas' short comment there points to the same direction as Carl's paper, but from a different angle. (I am not an economist, but it seems to me that Robin failed to address Johnicolas's points.)

Comment author: CarlShulman 04 May 2011 05:00:03AM 0 points [-]

Thanks for the plug!