Modest Superintelligences

Wei Dai

I'm skeptical about trying to build FAI, but not about trying to influence the Singularity in a positive direction. Some people may be skeptical even of the latter because they don't think the possibility of an intelligence explosion is a very likely one. I suggest that even if intelligence explosion turns out to be impossible, we can still reach a positive Singularity by building what I'll call "modest superintelligences", that is, superintelligent entities, capable of taking over the universe and preventing existential risks and Malthusian outcomes, whose construction does not require fast recursive self-improvement or other questionable assumptions about the nature of intelligence. This helps to establish a lower bound on the benefits of an organization that aims to strategically influence the outcome of the Singularity.

MSI-1: 10⁵ biologically cloned humans of von Neumann-level intelligence, highly educated and indoctrinated from birth to work collaboratively towards some goal, such as building MSI-2 (or equivalent)
MSI-2: 10¹⁰ whole brain emulations of von Neumann, each running at ten times human speed, with WBE-enabled institutional controls that increase group coherence/rationality (or equivalent)
MSI-3: 10²⁰ copies of von Neumann WBE, each running at a thousand times human speed, with more advanced (to be invented) institutional controls and collaboration tools (or equivalent)

(To recall what the actual von Neumann, who we might call MSI-0, accomplished, open his Wikipedia page and scroll through the "known for" sidebar.)

Building a MSI-1 seems to require a total cost on the order of $100 billion (assuming $10 million for each clone), which is comparable to the Apollo project, and about 0.25% of the annual Gross World Product. (For further comparison, note that Apple has a market capitalization of $561 billion, and annual profit of $25 billion.) In exchange for that cost, any nation that undertakes the project has a reasonable chance of obtaining an insurmountable lead in whatever technologies end up driving the Singularity, and with that a large measure of control over its outcome. If no better strategic options come along, lobbying a government to build MSI-1 and/or influencing its design and aims seems to be the least that a Singularitarian organization could do.

MSI-1: 10⁵ biologically cloned humans of von Neumann-level intelligence, highly educated and indoctrinated from birth to work collaboratively towards some goal, such as building MSI-2 (or equivalent)
MSI-2: 10¹⁰ whole brain emulations of von Neumann, each running at ten times human speed, with WBE-enabled institutional controls that increase group coherence/rationality (or equivalent)
MSI-3: 10²⁰ copies of von Neumann WBE, each running at a thousand times human speed, with more advanced (to be invented) institutional controls and collaboration tools (or equivalent)

(To recall what the actual von Neumann, who we might call MSI-0, accomplished, open his Wikipedia page and scroll through the "known for" sidebar.)

No Versailles, no National Socialist rise to power, no Hitler, no Holocaust
The Soviet Union is much smaller. It seems much less likley to dominate Eastern Europe and that part of the world is under German and Austria-Hungarian domination. Standards of living in Eastern Europe are as a result higher. Ukraine at the very least dosne't experience the Holodomor. Who knows perhaps Germans and Austrians even manage to do what the Entente couldn't and stamp out the Bolsheviks of Russia. In any case the mass slaughter and destruction of the Great Patriotic War are probably fully avoided.
Japanese expansion in the Far East is probably more limited, reducing the possibility of a Japanese-American war
National humiliation in France and Britian could perhaps result in either a Fascist or Communist take over. But both seem pretty unlikely. Even if they do happen, France in itself is far too weak for any revanchist warmongering, while Britain could only manage this if it was willing to lose its empire. And neither seem that likely to receive American support in such a war unless a US president does a FDR on steroids.
The Middle East remains under Ottoman control for the time being, I have a strong feeling that the Middle East several decades later in a alternative 2012 would be much less messed up than it currently is.

The immediate effects seem pretty darn beneficial and hard to beat. The end of the first world war with a Central powers victory basically changes the balance of power making Britain a second rate force decades earlier, preventing a Soviet rise to power, America remains much more isolationist... I have a very hard time seeing anything like a part two to that struggle.

A Stalin like figure might decide to try and invade central Europe but this seems unlikely. An American-Japanese war is still possible, but it seems unlikely to involve a European theatre in itself.

Did I mention we avoid the fucking holocaust and keep large chunks of Eastern Europe away and safe from Bolsheviks at least during their most damaging and bloodthirsty years? It just seems so overwhelmingly likely that this is a better world that I'm mystified why anyone would think it very likley to be worse.

Reducing the relevance of a global Communist vs. Capitalist struggle narrative also seems to much reduce the possibility of global annihilation. Maybe nukes are used in one or two ill though out wars, but then again nukes where used in our time line in a ill thought out war and we didn't turn out so bad.

Ah! You meant German victory! (or, at least, whatever the German masses would accept as victory or an honorable draw); I hadn't even thought about that; my first reaction was "So the French and the British bleed Germany dry on their own and impose an even harsher treaty", and I just went on from there. I'll have to consiider the above in detail, thanks a lot. This indeed sounds agreeable.

36

Modest Superintelligences

36

36

36

Modest Superintelligences

36

36