= 27d567bc2d48d9f40e9ccc20ea370b15)
wedrifid comments on What I've learned from Less Wrong - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (232)
The best essays will usually leave you with that impression. As will the best teachers.
Be careful. So will the less-than-best essays and teachers. It's a form of hindsight bias: you think this thing is obvious, but your thoughts were actually quite inchoate before that. A meme - particularly a parasitic meme - can get itself a privileged position in your head by feeding your biases to make itself look good, e.g. your hindsight bias.
When you see a new idea and you feel your eyes light up, that’s the time to put it in a sandbox - yes, thinking a meme is brilliant is a bias to be cautious of. You need to know how to take the thing that gave you that "click!" feeling and evaluate it thoroughly and mercilessly.
(I'm working on a post or two on the subject area of dangerous memes and what to do about them.)
I'm very interested in that, I think I need it. I just read this article about Mere Christianity by C. S. Lewis, and I was like "what the hell is wrong with me, that I didn't see at least some of those points myself?" It really scared me, and made me wonder what other nonsense I believe in, that I ought to have seen through right away...
The strength of C. S. Lewis's works seem to be that they were a whole lot less bad than the alternate sources of the same message.
It might be worth doing some analysis on the authoritative voice (the ability to sound right), and I speak as someone who's been a CS Lewis, GK Chesterton, Heinlein, Rand, and Spider Robinson fan. At this point, I suspect it's a pathology.
That's odd. I've been a fan of Heinlein and Spider Robinson but never Rand or Lewis. Haven't tried Chesterton.
You're actually the reason I started reading Spider Robinson.
This was over a year ago but I see that you're still around. I wanted to ask you more about this. How does Spider Robinson fit in with the others? I would also add Orwell, Kipling, and Christopher Hitchens. Maybe even Eliezer a bit.
A big part of it is that these authors talk about truth a lot and the harm of denying that it's there, and rail against and strawman other groups for refusing to accept the truth or even that truth exists.
What do you mean by a pathology? You think there was something wrong with those authors? Are you talking about overconfidence?
Spider Robinson is very definite and explicit about how things ought to be. Unfortunately, he extends this to the idea that people who are worth knowing like good jazz, Irish coffee, and puns.
I meant that there may be a pathology at my end-- being so fond of the authoritative voice that I could be a fan of writers with substantially incompatible ideas, and not exactly notice or care.
I suspect you may be reading his exaggerated enthusiasm for these things as a blanket statement about people who aren't worth knowing. For instance, I might, in a burst of excitement, say that people who don't like the song Waterfall aren't worth talking to, but I wouldn't mean it literally. It would be a figure of speech.
For instance, in one of the Callahan books he states (in the voice of the author, not as a character, IIRC) that if he had a large sum of money he'd buy everyone in the US a copy of "Running, Jumping, Standing Still" on CD because it would make the world so much better. I read this as hyperbole for how much he likes that CD, and I don't take it literally.
I may be misremembering or have missed something in his writing, though.
As far as you liking the voice, I doubt it's a pathology. I feel the same way you do and it's not surprising to me that a lot of people would find that kind of objectivity and confidence appealing. It is a bias, if you confuse the pleasure of reading those writers with their actual ideas, but since I vehemently disagree with most of the above writers I'm not too worried about it. (Do you still read or like those writers?)
I recently started rereading Atlas Shrugged, and was having fun with it-- no matter what else, Rand created a world where interesting things happen. It was also interesting because some things have changed. Her bad guy rich people were bad because they were slack-- they weren't interested in running their businesses, they had barely enough energy to get government favors. The modern type who's energetically taking as much money as possible out of the business with the intent of going somewhere else is barely present.
I can't stand Robinson any more. The tone of "we're cooler than the mundanes" has revolted me to the point where even the milder earlier version gets on my nerves. It's possible that I should give Stardance another chance some time. It's also possible that the effects of Very Bad Dreams have faded. Robinson has a sadistic imagination.
Back when, I bought a copy of Running Jumping Standing Still when I happened to see it, and was annoyed to find that I liked it.
I reread "Magic, Inc." recently, and liked it very much. I haven't read much Lewis or Chesterton lately.
My concern about pathology is a suspicion that what I like is the comfort of being told what to think in a palatable way.
I obviously haven't completely lost my taste for didactic fiction.
Second the confusion about this. I don't see what changed in him unless you mean the author's fictionalized daughter.
Yeah, that's one of the major criticisms of her book, that the poor honest robber-barons were being exploited by the mean old federal regulations, which has nothing to do with the real world.
I actually liked Anthem best of Rand's books, since it didn't pretend to take place in our world, but was set in a dystopian world instead.
You have to admit Rand can really write a page turner, even though her ideas are shit.
Heh, why were you annoyed that you liked Running Jumping Standing Still? You're opposed to music recommendations from writers?
I haven't read Stardance or Very Bad Dreams: what had the tone of being cooler than the mundanes, and what was the sadistic imagination? Why can't you stand him? I'm really not familiar with the tone you're talking about. The only tone that bothers me about SR is the whole "Let's be hippies and work everything out and it'll all be ok" thing. "Free Lunch" in particular. And his argument in one of the Callahan stories that his AGI character would have to be friendly because it wouldn't have human fear or insecurity. And have you read his "Night of Power"?
My favorite Heinlein are any of his short stories, and the novels Methuselah's Children, Time Enough for Love, To Sail Beyond the Sunset, The Cat Who Walked Through Walls, Number of the Beast, and The Moon is a Harsh Mistress.
As far as Lewis, you have to get past the religious stuff obviously, but I loved The Great Divorce.
I'm guessing you might like Robert Sheckley, who has some of the same "telling you what to think" but it's couched in extremely clever, biting satire. Sheer brilliance. He's SF's Mark Twain.
Are we both thinking of the book where vg gnxrf n qrhf rk znpuvan gb cerirag uhznavgl sebz qrfgeblvat vgfrys? Gur obbx va juvpu ng yrnfg bar punenpgre'f rkgencbyngrq ibyvgvba jbhyq cebonoyl qrfgebl uhznavgl, cnvashyyl?
Now the AI does seem absurd. I'm tempted to give SR a pass on that one because he had the characters talk about science fiction so much, they almost break the fourth wall to explain his motives. But the same author went on a rant elsewhere about the dangers of Star Trek science fantasy. His apparent exception for Callahan's seems a little forced.
I'll get the refrigerator.
I love Sheckley - but when does he tell you what to think? I read him when I was young, so maybe I didn't notice...?
One of the things I find incredibly interesting about Rand and her followers is that Rand is rather good at capturing the spirit of the envious and the bureaucratic, but not very good at making likeable heroes. They tend to be the Steve Jobs sort- it's nice that he exists somewhere far away from me and will sell me things, and he should be as unregulated as possible, but I'd rather not work for him or be his friend.
And so when I've gone to Objectivist meetings, most people there have the same hatreds and same resentments and feel them pretty strongly, but that seems to be the primary binding factor, rather than interest in rationality or personal kindness or shared goals. (I'm not counting everyone wanting to make a bunch of money for themselves as a shared goal.)
Rand looks like she's talking about production, but her real interest is in envy. And I agree with her that it's a terrible thing we shouldn't reward.
Night of Power was the alarm bell that made me realize Robinson was off the rails.
I'm not sure what the two of you mean by "Very Bad Dreams--" perhaps a misrecollection of "Very Bad Deaths?" If so, Very Bad Deaths is almost certainly the sadistic one.
Dude. AN ASSERTION IS PROVEN BY SOUNDING GOOD. It's a form of the Steve Jobs reality distortion superpower: come up with a viewpoint so compelling it will reshape people's perception of the past as well as the present.
(I must note that I'm not actually advocating this.)
Argument by assertion amusement from my daughter: "I'm running around the kitchen, but I'm not being annoying by running around the kitchen." An argument by assertion of rich depth, particularly from a three-year-old.
Did you ever get around to reading either of the papers I linked you to there btw?
Nuh. Still in the Pile(tm) with yer talk, which I have watched the first 5 min of ... I hate video so much.
Did you dislike your talk's content or your presentation? So far it looks like something that should be turned into a series of blog posts, complete with diagrams.
Neither really, it's the video itself I dislike. I've put the slides on Scribd, and I'm thinking of re-recording the soundtrack. Only trouble is, I'd have to watch the video first to remember what I said... and I hate video so much.
Hm, I'm a fan of Heinlein too, I guess I'd better not start reading those others. ;p Any idea where I can look for clues about the 'authoritative voice'?
The hard part with something like that not being how to question your ideas, but to notice that you have an idea that needs questioning. It's like reading Michael Behe's books on intelligent design and trying to understand the view inside his head, how a tenured biology professor could come up with such obvious-to-others defective arguments and fail to notice the low quality of his own thinking.
Given a clear explanation, it's more probably correct than secretly wrong. We don't live in a world dominated by true-sounding lies. Incorrect things should be generally more surprising than correct things, even if there are exceptions.
(It's confirmation bias, not hindsight bias. Hindsight bias is overestimation of prior probability upon observing a positive instance of an event.)
Less often. Learning bullshit is more likely to come with the impression that you are gaining sophistication. If something is so banal as to be straightforward and reasonable you gain little status by knowing it.
Yes, people have biases and believe silly things but things seeming obvious is not a bad sign at all. I say evaluate mercilessly those things that feel deep and leave you feeling smug that you 'get it'. 'Clicking' is no guarantee of sanity but it is better than learning without clicking.
Yes, I suspect I'm being over-cautious having been thinking about memetic toxic waste quite a lot of late. This suggests that when I'm describing the scary stuff in detail, I'll have to take care not to actually scare people out of both neophilia and decompartmentalisation.
That said, I recall the time I was out trolling the Scientologists and watched someone's face light up that way as she was being sold a copy of Dianetics and a communication course. She certainly seemed to be getting that feeling. Predatory memes - they're rare, but they exist.
Scary indeed. I suspect what we are each 'vulnerable' to will vary quite a lot from person to person.
Yes. I do think that a particularly dangerous attitude to memetic infections on the Scientology level is an incredulous "how could they be that stupid?" Because, of course, it contains an implicit "I could never be that stupid" and "poor victim, I am of course far more rational". This just means your mind - in the context of being a general-purpose operating system that runs memes - does not have that particular vulnerability.
I suspect you will have a different vulnerability. It is not possible to completely analyse the safety of an arbitrary incoming meme before running it as root; and there isn't any such thing as a perfect sandbox to test it in. Even for a theoretically immaculate perfectly spherical rationalist of uniform density, this may be equivalent to the halting problem.
My message is: it can happen to you, and thinking it can't is more dangerous than nothing. Here are some defences against the dark arts.
[That's the thing I'm working on. Thankfully, the commonest delusion seems to be "it can't happen to me", so merely scaring people out of that will considerably decrease their vulnerability and remind them to think about their thinking.]
This sort of thing makes me hope that the friendly AI designers are thinking like OpenBSD-level security researchers. And frankly, they need Bruce Schneier and Ed Felten and Dan Bernstein and Theo deRaadt on the job. We can't design a program not to have bugs - just not to have ones that we know about. As a subset of that, we can't design a constructed intelligence not to have cognitive biases - just not to have ones that we know about. And predatory memes evolve, rather than being designed from scratch. I'd just like you to picture a superintelligent AI catching the superintelligent equivalent of Scientology.
With the balancing message: Some people are a lot less vulnerable to believing bullshit than others. For many on lesswrong their brains are biassed relative to the population towards devoting resources to bullshit prevention at the expense of engaging in optimal signalling. For these people actively focussing on second guessing themselves is a dangerous waste of time and effort.
Sometimes you are just more rational and pretending that you are not is humble but not rational or practical.
I can see that I've failed to convince you and I need to do better.
In my experience, the sort of thing you've written is a longer version of "It can't happen to me, I'm far too smart for that" and a quite typical reaction to the notion that you, yes you, might have security holes. I don't expect you to like that, but it is.
You really aren't running OpenBSD with those less rational people running Windows.
I do think being able to make such statements of confidence in one's immunity takes more detailed domain knowledge. Perhaps you are more immune and have knowledge and experience - but that isn't what you said.
I am curious as to the specific basis you have for considering yourself more immune. Not just "I am more rational", but something that's actually put it to a test?
Put it this way, I have knowledge and experience of this stuff and I bother second-guessing myself.
(I can see that this bit is going to have to address the standard objection more.)
This is a failure mode common in when other-optimising. You assume that I need to be persuaded, put that as the bottom line and then work from there. There is no room for the possibility that I know more about my relative areas of weakness than you do. This is a rather bizarre position to take given that you don't even have significant familiarity with the wedrifid online persona let alone me.
It isn't so much that I dislike what you are saying as it is that it seems trivial and poorly calibrated to the context. Are you really telling a lesswrong frequenter that they may have security holes as though you are making some kind of novel suggestion that could trigger insecurity or offence?
I suggest that I understand the entirety of the point you are making and still respond with the grandparent. There is a limit to how much intellectual paranoia is helpful and under-confidence is a failure of epistemic rationality even if it is encouraged socially. This is a point that you either do not understand or have been careful to avoid acknowledging for the purpose of presenting your position.
I would be more inclined to answer such questions if they didn't come with explicitly declared rhetorical intent.
No, I'm actually interested in knowing. If "nothing", say that.
Regarding Scientology, I had the impression that they usually portray themselves to those they're trying to recruit as being like a self-help community ("we're like therapists or Tony Robbins, except that our techniques actually work!") before they start sucking you into the crazy?
Wait... did you just use Tony Robbins as the alternative to being sucked into the crazy?
I'm sure that whatever it is that Tony Robbins preaches is less crazy than the Xenu story. (Although Scientology doesn't seem any crazier than the crazier versions of mainstream religions...)
Here's a video in which he lays out what he sees as the critical elements of human motivation and action. Pay extra attention to the slides -- there's more stuff there than he talks about.
(It's a much more up-to-date and compact model than what he wrote in ATGW, by the way.)
One of Tony Robbins' books has been really helpful to me. Admittedly the effects mostly faded after the beginning, but applying his techniques put me into a rather blissful state for a day or two and also allowed for a period of maybe two weeks to a month during which I did not procrastinate. I also suspect I got a lingering boost to my happiness setpoint even after that. This are much better results than I've had from any previous mind-hacking technique I've used.
Fortunately I think I've been managing to figure out some of the reasons why those techniques stopped working, and have been on an upswing, mood and productivity-wise, again. "Getting sucked into the crazy" is definitely not a term I'd use when referring to his stuff. His stuff is something that's awesome, that works, and which I'd say everyone should read. (I already bought my mom an extra copy, though she didn't get much out of it.)
What book?