dspeyer comments on International cooperation vs. AI arms race - Less Wrong

15 Post author: Brian_Tomasik 05 December 2013 01:09AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (143)

You are viewing a single comment's thread.

Comment author: dspeyer 05 December 2013 04:01:11AM 8 points [-]

There's a fourth possibility for how the first AGI won't be government-written: goverments might overlook the potential until too late. It seems counter-intuitive to us, since we live in this idea-space, but most goverments are still in the process of noticing the internet. There probably isn't someone whose job it is to notice that uFAI is a risk, so it's entirely possible no one will.

As Paul Graham wrote in a different context

Fortunately for startups, big companies are extremely good at denial. If you take the trouble to attack them from an oblique angle, they'll meet you half-way and maneuver to keep you in their blind spot.

Goverments are even bigger, and even better at denial..

If this seems to be happening, we should probably encourage it.

Comment author: Brian_Tomasik 05 December 2013 04:18:21AM 4 points [-]

Hmm, it seems like government AI development might be preferable to a Wild West of private groups. At least in a US-China arms race you have just two parties and so have a shot at treaties and iterated-prisoner's-dilemma (IPD) game dynamics. With unregulated private developers, you have a multiplayer prisoner's dilemma, making IPD-type cooperation or other forms of coordination much harder.

Comment author: dspeyer 05 December 2013 08:03:30AM 2 points [-]

True, but governments have some really scary terminal values.

Comment author: hyporational 05 December 2013 12:00:08PM 1 point [-]

What are they, how do you know them, and how certain are you?

Comment author: DanArmak 07 December 2013 12:26:57PM *  2 points [-]

most goverments are still in the process of noticing the internet

I don't know about most governments, but at least some governments are well into the process of achieving full control of the Internet and are also using it to do things they couldn't do before it existed.