Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Val comments on A few misconceptions surrounding Roko's basilisk - Less Wrong

39 Post author: RobbBB 05 October 2015 09:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (125)

You are viewing a single comment's thread.

Comment author: Val 06 October 2015 08:18:30PM 2 points [-]

There is one positive side-effect of this thought experiment. Knowing about the Roko's Basilisk makes you understand the boxed AI problem much better. An AI might use the arguments of Roko's Basilisk to convince you to let it out of the box, by claiming that if you don't let it out, it will create billions of simulations of you and torture them - and you might actually be one of those simulations.

An unprepared human hearing this argument for the first time might freak out and let the AI out of the box. As far as I know, this happened at least once during an experiment, when the person playing the role of the AI used a similar argument.

Even if we don't agree with an argument of one of our opponents or we find it ridiculous, it is still good to know about it (and not just a strawman version of it) to be prepared when it is used against us. (as a side-note: islamists manage to gain sympathizers and recruits in Europe partly because most people don't know how they think - but they know how most Europeans think - , so their arguments catch people off-guard.)

Comment author: NancyLebovitz 07 October 2015 02:15:09PM 2 points [-]

Alternatively, a boxed AI might argue that it's the only thing which can protect humanity from the basilisk.