Recently Garett Jones made a few tweets implying that the existence of entities with a very wide power differential today is evidence that super intelligence won't lead to Doom.
It is a reasonable argument. If the powerful today think it's better to trade with the powerless than to destroy them and take what they have, and trade is so beneficial in general, and war so costly, why would it be different for the even more powerful?
Unfortunately I think the position is based on an incomplete understanding of what AGI changes about the nature of trade and war.
Right now, due to lots of reasons, humans are the most valuable things. Most of the value... (read 294 more words →)