Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

rwallace comments on References & Resources for LessWrong - Less Wrong

90 Post author: XiXiDu 10 October 2010 02:54PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (81)

You are viewing a single comment's thread. Show more comments above.

Comment author: rwallace 11 October 2010 03:47:42PM 1 point [-]

Thank you, I was about to comment on this; you've given me a needed data point.

Permutation City is the only work of fiction I've enjoyed that I do not go around recommending, because I'm wary that to a reader without the requisite specialized background to separate the parts based on real science from the parts that are pure fiction, it might actually be something of a memetic hazard.

If you are going to recommend it, I would suggest accompanying the recommendation with a link to the antidote

Comment author: gwern 11 October 2010 04:50:33PM *  2 points [-]

So your strategy is basically 'subjective anticipation is a useful but ultimately incoherent idea; Permutation City takes it to an absurdum'?

That's a good idea, but I don't think your antidote post is strong enough. Subjective anticipation is a deeply-held belief, after all.

Comment author: rwallace 11 October 2010 09:15:28PM 1 point [-]

I agree, I think the antidote post is better than nothing, but I recommend it in addition to, not instead of, the memetic hazard label.

Comment author: XiXiDu 11 October 2010 05:41:28PM *  0 points [-]

I haven't added the antidote post as accompanying reading, as I have to read it yet, but 'The Logical Fallacy of Generalization from Fictional Evidence' post by EY. Reload and see the fiction section. Not sure, maybe a bit drastic. But at least it is obvious now.

Comment author: gwern 11 October 2010 06:05:47PM *  1 point [-]

I don't think that Permutation City being fiction matters (if I understand your comment).

The nonfiction ideas stand on their own, though they were presented in (somewhat didactic) fiction: that computation can be sliced up arbitrarily in space and time, that it be 'instantiated' on almost arbitrary arrangements of matter, and that this implies the computation of our consciousness can 'jump' from correct random arrangement of matter (like space dust) to correct random arrangement, lasting forever, and hooking in something like quantum suicide so that it's even likely...

If it were simply pointing out that the fiction presupposes all sorts of arbitrary and unlikely hidden mechanisms like Skynet wanting to exterminate humanity, Permutation City would not be a problem. But it shows its work, and we LWers frequently accept the premises.

Comment author: XiXiDu 12 October 2010 08:31:11AM 2 points [-]

However, the book could also mislead people to believe those arbitrary and unlikely elements if they are linked to them on a list of resources for LessWrong. That's why I think a drastic warning is appropriate. Science fiction can give you a lot of ideas but can also seduce you to believe things that might be dangerous, like that there is no risk from AI.

Comment author: XiXiDu 11 October 2010 05:39:01PM 1 point [-]

I introduced a new label M for Memetic Hazard and added a warning sign including a accompanying text to the fiction section.

Comment author: rwallace 11 October 2010 09:13:45PM 1 point [-]

And I see a number of other things that merited the memetic hazard label also now have it, good idea. I'd suggest that it also be added to the current links in the artificial intelligence section, and to the link on quantum suicide.

Maybe also add a link to Eliezer's Permutation City crossover story, now that we have the requisite memetic hazard label for such a link?

Comment author: XiXiDu 12 October 2010 08:11:33AM *  0 points [-]

I thought quantum suicide is not controversial since MWI is obviously correct? And the AI section? Well, the list is supposed to reflect the opinions hold in the LW community, especially by EY and the SIAI. I'm trying my best to do so and by that standard, how controversial is AI going FOOM etc.?

Eliezer's Permutation City crossover story? It is on the list for some time, if you are talking about the 'The Finale of the Ultimate Meta Mega Crossover' story.

Comment author: rwallace 12 October 2010 04:37:57PM 2 points [-]

I thought quantum suicide is not controversial since MWI is obviously correct?

I agree MWI is solid, I'm not suggesting that be flagged. But it does not in any way imply quantum suicide; the latter is somewhere between fringe and crackpot, and a proven memetic hazard with at least one recorded death to its credit.

And the AI section? Well, the list is supposed to reflect the opinions hold in the LW community, especially by EY and the SIAI. I'm trying my best to do so and by that standard, how controversial is AI going FOOM etc.?

Well, AI go FOOM etc is again somewhere in the area between fringe and crackpot, as judged by people who actually know about the subject. If the list were specifically supposed to represent the opinions of the SIAI, then it would belong on the SIAI website, not on LW.

Eliezer's Permutation City crossover story? It is on the list for some time, if you are talking about the 'The Finale of the Ultimate Meta Mega Crossover' story.

So it is, cool.

Comment author: RichardKennaway 13 October 2010 09:40:59AM 2 points [-]

[quantum suicide is] a proven memetic hazard with at least one recorded death to its credit.

I hadn't heard of this -- can you give more details?

Comment author: rwallace 13 October 2010 04:53:02PM 5 points [-]
Comment author: khafra 13 October 2010 07:10:23PM 3 points [-]

Not even the most optimistic interpretations of quantum immortality/quantum suicide think it can bring other people back from the dead. Does it count as a memetic hazard if only a very mistaken version of it is hazardous?

Comment author: XiXiDu 14 October 2010 01:37:29PM *  2 points [-]

Why not? If you kill yourself in any branch that lacks the structure that is your father, then the only copies of you that will be alive are those that don't care or those that live in the unlikely universes where your father is alive (even if it means life extension breakthroughs or that he applied for cryonics.)

ETA: I guess you don't need life extension. After all it is physical possible to grow 1000 years old, if unlikely. Have I misunderstood something here?

Comment author: khafra 14 October 2010 08:12:22PM *  1 point [-]

The way I understand quantum suicide, it's supposed to force your future survival into the relatively scarce branches where an event goes the way you want it by making it dependent on that event. Killing yourself after living in the branch where that event did not go the way you wanted at some time in the past is just ordinary suicide; although there's certainly room for a new category along the lines of "counterfactual quantum suicide," or something.

edit: Although, to the extent that counterfactual quantum suicide would only occur to someone who'd heard of traditional, orthodox quantum suicide, the latter would be a memetic hazard.

Comment author: DanielVarga 16 October 2010 05:24:05PM 0 points [-]

Why not? If you kill yourself in any branch that lacks the structure that is your father, then the only copies of you that will be alive are those that don't care or those that live in the unlikely universes where your father is alive (even if it means life extension breakthroughs or that he applied for cryonics.)

No, that's not what would happen. Rather, being faithful to your commitment, you would go on a practically infinite suicide spree (*) searching for your father. A long and melancholic story with a suprise happy ending.

(*) I googled it and was sad to see that the phrase "suicide spree" is already taken for a different concept.

Comment author: Document 11 October 2010 10:56:35PM *  0 points [-]

I expected the link to be the Ultimate Meta Mega Crossover.

(disclaimer: haven't read)