jacob_cannell comments on Should I believe what the SIAI claims? - Less Wrong

23 Post author: XiXiDu 12 August 2010 02:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (600)

You are viewing a single comment's thread. Show more comments above.

Comment author: jacob_cannell 01 September 2010 09:27:13PM *  0 points [-]

I for one largely agree, but a few differences:

Artificial systems didn't produce anything a century ago; even without a strong exponential, they're clearly getting somewhere.

We've had a strong exponential since the beginning of computing. Thinking that humans create computers is something of a naive anthropocentric viewpoint: humans don't create computers and haven't for decades. Human+computer systems create computers, and the speed of progress is largely constrained by the computational aspects even today (computers increasingly do more of the work, and perhaps already do the majority). To understand this more, read this post from a former intel engineer (and apparently AI lab manager). Enlightening inside knowledge, but for whatever reason he only got up to 7 karma and wandered away.

Also, if you plotted out the data points of brain complexity on earth over time, I'm near certain it also follows a strong exponential.

The differences between all these exponentials are 'just' constants.

The vast majority of possible "wants" done thoroughly will destroy us. (Any goal taken to extremes will use all available matter in accomplishing it.)

I find this dubious, mainly because physics tells us that using all available matter is actually highly unlikely to ever be a very efficient strategy.

However, agreed about the potential danger of future hyper-intelligence.