Often, I find arguments where people are assumed to be rational

Bias

Often the desirable outcome of an action or 'strategy' is taken to have been the goal that the actor deliberately attempted to attain.

Diamond in a box:

Suppose you're faced with a choice between two boxes, A and B. One and only one of the boxes contains a diamond. You guess that the box which contains the diamond is box A. It turns out that the diamond is in box B. Your decision will be to take box A. I now apply the term volition to describe the sense in which you may be said to want box B, even though your guess leads you to pick box A.

Let's say that Fred wants a diamond, and Fred asks me to give him box A. I know that Fred wants a diamond, and I know that the diamond is in box B, and I want to be helpful. I could advise Fred to ask for box B instead; open up the boxes and let Fred look inside; hand box B to Fred; destroy box A with a flamethrower; quietly take the diamond out of box B and put it into box A; or let Fred make his own mistakes, to teach Fred care in choosing future boxes.

But I do not simply say: "Well, Fred chose box A, and he got box A, so I fail to see why there is a problem." There are several ways of stating my perceived problem:

Fred was disappointed on opening box A, and would have been happier on opening box B.

It is possible to predict that if Fred chooses box A, Fred will look back and wish he had chosen box B instead; while if Fred chooses box B, Fred will be satisfied with his choice.

Fred wanted "the box containing the diamond", not "box A", and chose box A only because he guessed that box A contained the diamond.

If Fred had known the correct answer to the question of simple fact, "Which box contains the diamond?", Fred would have chosen box B.

Hence my intuitive sense that giving Fred box A, as he literally requested, is not actually helping Fred.

If you find a genie bottle that gives you three wishes, it's probably a good idea to seal the genie bottle in a locked safety box under your bed, unless the genie pays attention to your volition, not just your decision.

--CEV

it appears to me that human beings are emotional, situationally defined social animals, much more than they are rational actors

You imply that there is a standard of rationality people are deviating from. Yes?

Hi Kouran, and welcome.

Your critique of "rationalism" as you currently understand it is, I think, valid. The goal of LessWrong, as I understand it (though I'm no authority, I just read here sometimes), is to help people become more rational themselves. As thomblake has already pointed out, we tend to believe with you in the general irrationality of humans. We also believe that this is a sort of problem to be fixed.

However, I also think you're being unfair to people who use the Rationality Assumption in economics, biology or elsewhere. You say that:

Often the desirable outcome of an action or 'strategy' is taken to have been the goal that the actor deliberately attempted to attain.

That's not an assumption that the theory requires. The Rationality Assumption only requires us to interpret the actions of an agent in terms of how well it appears to help it fulfill its goals. It needn't be conscious of such "goals". This type of goal is usually referred to as a revealed preference. Robin Hanson at Overcoming Bias, a blog that's quite related to LessWrong, also loves pointing out and discussing the particular problem that you've raised. He usually refers to it as the "Homo hypocritus hypothesis". You might enjoy reading some related entries on his blog. The gist of the distinction I'm trying to point to is actually pretty well-summarized by Joe Biden:

My dad used to have an expression: "Don't tell me what you value. Show me your budget, and I'll tell you what you value."

It's my own humble opinion that economists occasionally make the naive jump from talking about revealed preferences to talking about "actual" preferences (whatever those may be). In such cases, I agree with you that a disposition toward "rationalism" could be dangerous. But again, that's not the accepted meaning of the word here. I also think it might be just as naive to take peoples' stated preferences (whether stated to themselves or others) to be their "actual" preferences.

There have been attempts on LW to model the apparent conflict between the stated preferences and revealed preferences of agents, my favourite of which was "Conflicts Between Mental Subagents: Expanding Wei Dai's Master-Slave Model". If I were to taboo the word "rationality" in explaining the goal of this site, I'd say the goal is to help people bring these two mental sub-agents into agreement; to avoid being a Homo hypocritus; to help one better synchronize their stated preferences with their revealed preferences.

Clearly, the meanings of the word "rationality" that you have, and that this community has, are related. But they're not the same. My goal in linking to the several articles in the above text, is to help you understand what is meant by that word here. Good luck and I hope you find the discourse you're looking for!

That's just about right. Humans are massively irrational; but we tend to regard that as a bug and work to fix it in ourselves.

It sounds like the Straw Vulcan talk might be relevant to some of your thoughts on rationality and emotion...

Hello Less Wrong community, I am Kouran.

What follows may be a bit long, and maybe a little dramatic. I'm sorry if that is uncourteous, still I feel the following needs saying early on. Please bear with me.

I'm a recently be-bachelored sociologist from the Netherlands, am male and in my early twenties. I consider myself a jack of several trades – among them writing, drawing, cooking and some musical hobbies – but master of none. However, I do entertain the hope that the various interests and skills add up to something effective, to my becoming someone in a position to help people who need it, and I intend to take action to approach this end.

I found Less Wrong through the intriguing Harry Potter fanfiction story called 'the Methods of Rationality.' The story entertains me greatly, and the more abstract themes stimulate me and I find myself wishing to enter discussions regarding these matters. Instead of bothering the author of the story I decided to have a look here instead. Please note that I write this before having read any of the Sequences and only a few smaller articles. I intend to get on that soon, but as introductions go I feel it is better to present myself first. I hope you will forgive any offense the following section may give.

My relation to rationalism is quite strained. I am more often in the position where I have to attack theories mostly concerned with rationality, than that I have to defend them. Often, I find arguments where people are assumed to be rational and to make informed choices are classist and uncritical of the way people are shaped by society and vice versa. Often the desirable outcome of an action or 'strategy' is taken to have been the goal that the actor deliberately attempted to attain. Often this is done at the cost of more likely explanations that make fewer unfounded assumptions. I do not at all mean that Less Wrong is implicated in this, in fact: I hope I am right to believe that quite the opposite is being attempted here, my point is that I am more used to denying people's rationality in arguments than invoking it as a way to explain social life.

That is not to say I deny that people can engage in rational thought. Rather, it appears to me that human beings are emotional, situationally defined social animals, much more than they are rational actors. Rational thought, as I see it, is something that occurs in certain relatively rare circumstances. And when it occurs it is always bound to people's social, emotional, physical lives. Often it is group membership and identification, rather than a objective calculation of merit, that defines the outcome of a deliberation, when a deliberation even takes place at all.

So then why am I here? For one thing, I would like to discuss these ideas with people who are knowledgeable about them, but who are also tolerant enough of dissidence that they'll do so in a relaxed and well, rational, way. For another, I believe that more rationality, as truly rational as we can make it, will help our species get through the ages and improve upon the fate of it´s members and the other beings it dominates. The Methods of Rationality and what little I´ve seen of this community has led me to believe that, despite having a perspective that differs from mine, people at Less Wrong are aware of some of the ways in which people are inherently not rational. That rationality is something that needs to be promoted and created, not something that is already the dominant cause of human action. For a third, I cannot deny that I am a person who engages in a lot of thinking. Despite differing perspectives I believe this community may be able to help me develope. My ´story of origin´, if I am to present myself to you as a rationalist, involves a change in my views regarding the false or harmfull style of rationalism I mentioned earlier in this post. I once struggled with the idea that rationalism itself is to blame for perceived injustice and failings of modernity. But at some point I came to the conclusion that this is not the case. At fault is not a human rationality that will forever remain at odds with our emotions, and with those people who were not sufficiently introduced to rationality. People should be able to deliberate rationally while understanding that most of their being is disinclined to yield to abstract models and lofty humanist ideals. At fault is not rationalism or the imperfection of our brians, but incomplete and erroneous rationalism that is employed to serve people who have no need or appreciation for a critical eye cast upon themselves.

I think the community of Less Wrong is very right to consider human rationality an art.

I thank you for your patience,

– Kouran.