Rain comments on Should I believe what the SIAI claims? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (600)
Thank you for continuing to engage my point of view, and offering your own.
That's an interesting hypothesis which easily fits into my estimated 90+ percent bucket of failure modes. I've got all kinds of such events in there, including things such as, there's no way to understand intelligence, there's no way to implement intelligence in computers, friendliness isn't meaningful, CEV is impossible, they don't have the right team to achieve it, hardware will never be fast enough, powerful corporations or governments will get there first, etc. My favorite is: no matter whether it's possible or not, we won't get there in time; basically, that it will take too long to be useful. I don't believe any of them, but I do think they have solid probabilities which add up to a great amount of difficulty.
But the future isn't set, they're just probabilities, and we can change them. I think we need to explore this as much as possible, to see what the real math looks like, to see how long it takes, to see how hard it really is. Because the payoffs or results of failure are in that same realm of 'astronomical'.