= 27d567bc2d48d9f40e9ccc20ea370b15)
XiXiDu comments on An inflection point for probability estimates of the AI takeoff? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (45)
I do not doubt that humans can create superhuman AI, but I don't know how likely self-optimizing AI is. I am aware of the arguments. But all those arguments rather seem to me like theoretical possibilities, just like universal Turing machines could do everything a modern PC could do and much more. But in reality that just won't work because we don't have infinite tapes, infinite time...
Applying intelligence to itself effectively seems problematic. I might just have to think about it in more detail. But intutively it seems that you need to apply a lot more energy to get a bit more complexity. That is, humans can create superhuman intelligence but you need a lot of humans working on it for a long time and have a lot of luck stumbling upon unknown unknowns.
It is argued that the mind-design space must be large if evolution could stumble upon general intelligence. I am not sure how valid that argument is, but even if that is the case, shouldn't the mind-design space reduce dramatically with every iteration and therefore demand a lot more time to stumble upon new solutions?
Another problem I have is that I don't get why people here perceive intelligence to be something proactive with respect to itself. No doubt there exists some important difference between evolutionary processes and intelligence. But if you apply intelligence to itself, this difference seems to diminish. How so? Because intelligence is no solution in itself, it is merely an effective searchlight for unknown unknowns. But who knows that the brightness of the light increases proportionally with the distance between unknown unknowns? To have an intelligence explosion the light would have to reach out much farther with each generation than the increase of the distance between unknown unknowns...I just don't see that to be a reasonable assumption.
What appears to be a point against the idea:
This is from: Is there an Elegant Universal Theory of Prediction?
This is from your link.
But if it can be predicted by a trivial algorithm, it has LOW Kolmogorov complexity.
Check with definition 2.4. In the technical sense used in the document, a predictor is not defined as being something that outputs the sequence - it is defined as something that eventually learns how to predict the sequence - making at most a finite number of errors.
Strings with high Kolmogorov complexity being "predicted" by trivial algorithms is quite compatible with this notion of "prediction".
So, above the last wrongly predicted output, the whole sequence is as complex as the (improved) predictor?
Here's an example from the paper that helps illustrate the difference: if the sequence is a gigabyte of random data repeated forever, it can be predicted with finitely many errors by the simple program "memorize the first gigabyte of data and then repeat it forever", though the sequence itself has high K-complexity.
No it has not. The algorithm for copying the first GB forever is small and the Kolmogorov's complexity is just over 1GB.
For the entire sequence.
Yes, but the predictor's complexity is much lower than 1GB.
The paper also gives an example of a single predictor that can learn to predict any eventually periodic sequence, no matter how long the period.
Predictor should remember what happened. It has learned. Now it's 1 GB heavy.
It looks like you just dislike the definitions in the paper and want to replace them with your own. I'm not sure there's any point in arguing about that.