David_Gerard comments on Making your explicit reasoning trustworthy - Less Wrong

82 Post author: AnnaSalamon 29 October 2010 12:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (93)

You are viewing a single comment's thread. Show more comments above.

Comment author: David_Gerard 01 November 2010 12:37:16AM *  15 points [-]

Well, yeah. Scientology is sort of the Godwin example of dangerous infectious memes. But I've found the lessons most useful in dealing with lesser ones, and it taught me superlative skills in how to inspect memes and logical results in a sandbox.

Perhaps these have gone to the point where I've recompartmentalised and need to aggressively decompartmentalise again. Anna Salamon's original post is IMO entirely too dismissive of the dangers of decompartmentalisation in the Phil Goetz post, which is about people who accidentally decompartmentalise memetic toxic waste and come to the startling realisation they need to bomb academics or kill the infidel or whatever. But you always think it'll never happen to you. And this is false, because you're running on unreliable hardware with all manner of exploits and biases, and being able to enumerate them doesn't grant you immunity. And there are predators out there, evolved to eat people who think it'll never happen to them.

My own example: I signed up for a multi-level marketing company, which only cost me a year of my life and most of my friends. I should detail precisely how I reasoned myself into it. It was all very logical. The process of reasoning oneself into the mouth of a highly evolved predator tends to be. The cautions my friends and family gave me were all heuristic. This was before I studied Scientology in detail, which would I suspect have given me some immunity.

I should write a post on the subject (see my recent comments) except Anna's post covers quite a lot of it.

Comment author: CarlShulman 02 November 2010 02:19:12PM 3 points [-]

I would be interested in reading this, and especially about what caused the initial vulnerability.

Comment author: NancyLebovitz 01 November 2010 05:17:17AM 6 points [-]

I hope you'll also post about how you reasoned yourself out of it.

Comment author: David_Gerard 01 November 2010 11:49:11AM *  5 points [-]

Reading the sucker shoot analogy in a Florence Littauer book (CAUTION: Littauer is memetic toxic waste with some potentially useful bits). That was the last straw after months of doubts, the bit where it went "click! Oh, this is actually really bad for me, isn't it?" Had my social life been on the internet then (this was 1993) this would have been followed with a "gosh, that was stupid, wasn't it?" post. I hope.

It may be relevant that I was reading the Littauer book because Littauer's books and personality theories were officially advocated in the MLM in question (Omegatrend, a schism of Amway) - so it seemed to be coming from inside. I worry slightly that I might have paid insufficient attention had it been from outside.

I'd be interested to know how others (a) suffered a memetic cold (b) got out of it. Possible post material.

Comment author: CarlShulman 26 April 2011 05:19:03AM 4 points [-]

Just re-read this thread, and I'm still keen to hear how you reasoned yourself into it.