Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

gwern comments on References & Resources for LessWrong - Less Wrong

90 Post author: XiXiDu 10 October 2010 02:54PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (81)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 11 October 2010 04:50:33PM *  2 points [-]

So your strategy is basically 'subjective anticipation is a useful but ultimately incoherent idea; Permutation City takes it to an absurdum'?

That's a good idea, but I don't think your antidote post is strong enough. Subjective anticipation is a deeply-held belief, after all.

Comment author: rwallace 11 October 2010 09:15:28PM 1 point [-]

I agree, I think the antidote post is better than nothing, but I recommend it in addition to, not instead of, the memetic hazard label.

Comment author: XiXiDu 11 October 2010 05:41:28PM *  0 points [-]

I haven't added the antidote post as accompanying reading, as I have to read it yet, but 'The Logical Fallacy of Generalization from Fictional Evidence' post by EY. Reload and see the fiction section. Not sure, maybe a bit drastic. But at least it is obvious now.

Comment author: gwern 11 October 2010 06:05:47PM *  1 point [-]

I don't think that Permutation City being fiction matters (if I understand your comment).

The nonfiction ideas stand on their own, though they were presented in (somewhat didactic) fiction: that computation can be sliced up arbitrarily in space and time, that it be 'instantiated' on almost arbitrary arrangements of matter, and that this implies the computation of our consciousness can 'jump' from correct random arrangement of matter (like space dust) to correct random arrangement, lasting forever, and hooking in something like quantum suicide so that it's even likely...

If it were simply pointing out that the fiction presupposes all sorts of arbitrary and unlikely hidden mechanisms like Skynet wanting to exterminate humanity, Permutation City would not be a problem. But it shows its work, and we LWers frequently accept the premises.

Comment author: XiXiDu 12 October 2010 08:31:11AM 2 points [-]

However, the book could also mislead people to believe those arbitrary and unlikely elements if they are linked to them on a list of resources for LessWrong. That's why I think a drastic warning is appropriate. Science fiction can give you a lot of ideas but can also seduce you to believe things that might be dangerous, like that there is no risk from AI.