PhilGoetz comments on Exploitation and cooperation in ecology, government, business, and AI - Less Wrong

18 Post author: PhilGoetz 27 August 2010 02:27PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (43)

You are viewing a single comment's thread. Show more comments above.

Comment author: jacob_cannell 28 August 2010 03:25:01AM 2 points [-]

Any artificial intelligence will have internal structure. Artificial intelligences, unlike humans, do not come in standard-sized reproductive units, walled off computationally; there is therefore no reason to expect individuals to exist in a post-AI society. But the bulk of the computation, and hence the bulk of the potential consciousness, will be within small, local units (due to the ubiquity of power-law distributions, the efficiency of fractal transport and communication networks, and the speed of light).

Physics is local. The speed of light is a derivative of that general principle. The local nature of our universe implies some strict limits on intelligence. Curiously, it looks like the only way to transcend these limits (to get a really powerful single intelligence/computer) is to collapse into a black hole, at which point you necessarily seal yourself off and give up any power in this universe. Interesting indeed.

But I have no idea how you leap to the conclusion "there is therefore no reason to expect individuals to exist in a post-AI society." Although partly because I dont know what a post-AI society is. I understand post-human .. but post-AI? Is that the next thing after the next thing? That seems to be getting ahead of ourselves.

Also, you seem to reach the conclusion that there will not necessarily be any individuality in the 'post-AI' future society, but then give several good reasons why such individuality may persist. (namely, speed of light, locality of physics)

But what is individuality? One could say that we are a global consciousness today with just the "bulk of computation" in "small, local units".

Comment author: PhilGoetz 28 August 2010 03:27:14PM *  0 points [-]

Also, you seem to reach the conclusion that there will not necessarily be any individuality in the 'post-AI' future society, but then give several good reasons why such individuality may persist. (namely, speed of light, locality of physics)

Yes - it's a matter of degree. Humans are isolated to a greater extent than I expect AIs to be. Also, the word "individual" means (I think) non-divisible, which AIs probably will not be.

I agree, the term post-AI is confusing. I'll remove the 'post'.