GuySrinivasan comments on New report: Intelligence Explosion Microeconomics - Less Wrong

45 Post author: Eliezer_Yudkowsky 29 April 2013 11:14PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (244)

You are viewing a single comment's thread. Show more comments above.

Comment author: GuySrinivasan 30 April 2013 06:44:55PM 0 points [-]

This is in the context of reinvesting dividends of cognitive work, assuming it takes exponentially greater investments to produce linearly greater returns. For example, maybe we get a return of log(X) cognitive work per time with what we have now, and to get returns of log(X+k) per time we need to have invested X+k cognitive work. What does it look like to reinvest all of our dividends? After dt, we have invested X+log(X) and our new return is log(X+log(X)). After 2dt, we have invested X+log(X)+log(X+log(X)), etc.

The corrected paragraph would then look like:

Therefore, an AI trying to invest an amount of cognitive work w to improve its own performance will get returns that go as log(w), or if further reinvested, an additional log(1+log(w)/w), and the sequence log(w)+log(1+log(w)/w)+log(1+log(w+log(w))/(w+log(w))) will converge very quickly.

Except then it's not at all clear that the series converges quickly. Let's check... we could say the capital over time is f(t), with f(0)=w, and the derivative at t is f'(t)=log(f(t)). Then our capital over time is f(t)=li^(-1)(t+li(w)). This makes our capital / log-capital approximately linear, so our capital is superlinear, but not exponential.