RobbBB comments on Why I'm Skeptical About Unproven Causes (And You Should Be Too) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (102)
Huh?
Is not embarrassing yourself by looking a bit like Ted Kaczynski your only terminal value?
I suggest thinking inside the box more. The hypothetical is 'What if FAI is impossible?', not 'What if we can't prove that FAI is actual?'. But all your suggestions are attempts to resist that premise, not attempts to explore its consequences. One of the basic concerns of the EA movement is that it's very dangerous not to be able to seriously entertain worst-case scenarios.
More, But there's still a chance, right? isn't the right way to think about any question. The question is whether the probability is low enough to be worth the risk, not whether the probability is nonzero.
If it kills us all in less than the time it takes to construct a reasoned paragraph in any human language -- which is possibly the default scenario, the most common one -- then that will be difficult.
More importantly, any relevant argument we could come up with will almost certainly have already been thought of by the UFAI, and presented with far more rigor than any human could. You should be more confident that a chimpanzee could beat Garry Kasparov in chess than that a human could outwit a post-FOOM AI.