waveman comments on Dark Arts of Rationality - Less Wrong

136 Post author: So8res 19 January 2014 02:47AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (185)

You are viewing a single comment's thread. Show more comments above.

Comment author: jimmy 16 January 2014 10:11:50AM *  25 points [-]

Anyone that know's me knows that I'm quite familiar with the dark arts. I've even used hypnosis to con Christians into Atheism a half dozen times. The tempting idea is that dark arts can be used for good - and that the ends justify the means. I've since changed my mind.

The thing is, even though I don't advocate dark arts for persuasion let alone rationality, I almost entirely agree with the actions you advocate. I just disagree strongly with the frame through which you look at them.

For example, I am heavily into what you call "changing terminal goals", however I disagree that I'm changing terminal goals. If I recognize that pursuing instrumental goal A for sake of "terminal" goal B is the best way to achieve goal B, I'll self modify in the way you describe. I'll also do that thing you frame as "being inconsistent" where I make sure to notice if chasing goal A is no longer the best way to achieve goal B, I self modify to stop chasing goal A. If you make sure to remember that step, goals are not sticky. You chase goal A "for its own sake" iff it is the best way to achieve goal B. That's what instrumental goals are.

The way I see it, the difference in motivation comes not from "terminal vs instrumental", but from how you're focusing your attention. In what you call "instrumental" mode, you aren't focusing solely on your instrumental goal. You're trying to work on your instrumental goal while you keep glancing over at your terminal goal. That's distracting, and of course it doesn't work well. If it's a long term goal of course you don't see immediate improvements - and so of course you lose motivation. What you call "hacking my goals to be terminal" I call "realizing at a gut level that in order to get what I want, I need to focus on this instrumental goal without expecting immediate results on my terminal goal"

But there are also downsides to allowing yourself to "fool yourself". In particular, through that frame, the thought is "it's false, but so what? It's useful!". That stops curiosity dead when you should be asking the question "if it's false, why is it so useful? Where's the mutual information that allows it to function as a control system?" and "what true beliefs do better?".

For example, your "nothing is beyond my grasp" belief. It's empowering, sure. Just because you recognize that it isn't technically true doesn't mean you should deprive yourself of that empowerment - of course. However, lying isn't necessary for that empowerment. The problem isn't that you believe you're defeat-able. The problem is that you fear failure. So instead of focusing on the task at hand, you keep glancing over at the possibility of failure when you should be keeping your eyes on the road. One of the big take home lessons from studying hypnotism is that It's always about direction of attention. Strip away the frames and motivations and look at where the attention is.

My version or your empowering belief is (to try to crudely translate into words) "I want to succeed. I might not, and if I don't, it will be truly disappointing. And that's okay. And even though I might fail, I might not and that would be truly amazing. So I'm going to throw myself at it without looking back". And my version is better. My version is more stable under assault.

My wrestling coach would spout the cliche "If you can't believe you'll win, you wont!". If I had bought into that, the moment reality slaps me in the face I'd lose grasp of my delusion and crumble. Instead, I laughed at the idea. I went into matches already accepting defeat and focusing on winning anyway - and it allowed me to win a few matches that no one thought I could possibly win.

Comment author: waveman 01 February 2014 10:26:23AM *  5 points [-]

lying isn't necessary for that empowerment

I am not sure about that. In any case some 'hack' is required. Merely knowing the right thing to do is not sufficient.

More broadly, the LW community has not really come to terms with the deep emotionality of the human brain. Yes we pay lip service to cognitive biases etc, but the assumption is more or less that if we gain rationality knowledge and skills we will be more rational.

This assumes our rational brain is in charge. In reality we are a fairly intelligent cognitive brain (average IQ on LW high 130s), with various well known limitations, tightly coupled to an emotional 4 year old. And that four year old can make us not see or notice what it doesn't want us to know. It can make us forget things when it wants. It can control what we pay attention to and what we ignore. It can direct us to come up with elegant rationalisations for whatever it wants us to do.

Take something as simple as weight loss. The solution for most people [not everyone] is actually very simple - each fewer calories of nutrient rich food and exercise moderately. Track your weight and adjust the amount of food and exercise accordingly. Many really smart people struggle to execute this.

So you try to lose weight and what happens? You suddenly realize that you "forgot" your diet and ate a whole lot of food. Or you "realize" that when you try to lose weight you cannot concentrate. Also how irritable I get. Etc. My weight loss breakthrough occurred when I realized that a lot of the "problems" that happened when on weight loss diets were manufactured ie manipulation by my unconscious so it would get its food. As far as it knew, I was in a famine and I needed urgently to get some food, as much as possible actually. I needed to come to an accommodation with my unconscious and get it on board.

My commitment to it is a) I will stop dieting when I get to a healthy weight b) I will eat a nutrient rich diet c) I will reward myself for losing weight in various ways such as taking nootropics that I cannot take when very overweight due blood pressure side effects etc. So I think we are in it together now.

I think we as LWers need to take on board the whole issue of our tightly coupled unconscious and deal with it in a systematic way. The top post and above posts here are a start. I hope there will be a lot more to come.

It is interesting reading this whole thread how many of the gimmicks and tricks that we are talking about have a very long history. I wonder how much apparent irrationality is actually one of these gimmicks. They are irrational when looked at on their own, but in the context of the makeup of our brains they may make perfect sense.