Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

JoshuaZ comments on Q&A with Michael Littman on risks from AI - Less Wrong Discussion

15 Post author: XiXiDu 19 December 2011 09:51AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (88)

You are viewing a single comment's thread. Show more comments above.

Comment author: JoshuaZ 19 December 2011 04:25:11PM *  7 points [-]

And we probably suck at these tasks about as much as it's possible to suck and still build a technological civilization, because otherwise we would have built it at an earlier point in our evolution.

I don't think this follows. Humans spent thousands of years in near stagnation (the time before the dawn of agriculture is but one example). It isn't clear what caused technological civilization to take off but when new discoveries occurred almost looks like some sort of nearly random process except that the probability of a new discovery or invention increases as more discoveries occur. I'd almost consider modeling it as a biased coin which starts off with an extreme bias towards tails, but each time it turns up heads, the bias shifts a bit in the heads direction. Something like P(heads on the nth flip)= (1+k)/(10^5 + k) where k is the number of previous flips that came up heads.If that's the case, then the timing doesn't by itself tell us much about where our capacity is for civilization. It doesn't look that improbable that some other extinct species might even have had the capability at about where we are or higher but went extinct before they got those first few lucky coin flips.

Comment author: billswift 19 December 2011 06:31:20PM 3 points [-]

almost looks like some sort of nearly random process except that the probability of a new discovery or invention increases as more discoveries occur.

And as population increases that would tend to increase the rate of discovery or invention as well. This is basically Julian Simon's argument in The Great Breakthrough and Its Causes, that gradually increasing population hit a point where the rates of discovery and invention suddenly started increasing rapidly (and population then started increasing even more rapidly), resulting in the Renaissance and ultimately in the Industrial Revolution. He gives some thought and argument as to why they didn't happen earlier in India or China, but I think the specific arguments a bit iffy.

Comment author: Vladimir_Nesov 19 December 2011 05:44:08PM *  0 points [-]

In a blink of evolution's eye.