= 27d567bc2d48d9f40e9ccc20ea370b15)
abramdemski comments on Q&A with Abram Demski on risks from AI - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (70)
It's not just how much space there is, it's also how easy it is to reach. I think there is a lot of space, but that it requires a lot of computing power to reach. If a system is merely superhuman, then destroying humankind will not be a good strategy. A system would have to climb quite high before cooperation with humans started to look suboptimal. (There are also Mark Waser's arguments that cooperation looks more and more optimal as a system gets smarter, but he hasn't established that mathematically...)
I understand your point. We agree that there is a lot of space above, we don't agree how easy it is to climb up there.
With a LOT of CPU, it would be almost trivial. But what can we do (what can AI do) with much less?
We have done many amazing things in mathematics mostly at the state of the CPU starvation. We have accomplished a lot with a very limited calculation resources. So much so, that we assumed how "thinking is not equal calculating".
A substantial CPU usage optimization may be possible for an AI and by an AI. I guess.
One question which is pertinent here: is there anything dramatically better than Levin search for general problem solving? (That's a real question, not intended rhetorically.)
Some approximate method could be as nearly as good and a lot cheaper to compute. I am at least 60% confident it exists.
Some highly optimized implementation might be acceptable. Only 30% confident that there is one such.
For the case, that there is something different and radically better, I give 25%.
And there is another optic. Say, that I have 10^20 CPU cycles every second. How much of a Levin search I can afford with this? Maybe it is enough. I give 80% chance for this one.
What if I have only 10^15 cps and still enough? I give 50% chance for this.
And maybe even 10^10 cps is quite all right. So that I could do it today, had been smarter. I give it 10%.
Of course, those are all wild guesses, but it's my best answer I can give.