pjeby comments on A social norm against unjustified opinions? - Less Wrong

11 Post author: Kaj_Sotala 29 May 2009 11:25AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (158)

You are viewing a single comment's thread. Show more comments above.

Comment author: pjeby 30 May 2009 03:09:47PM *  5 points [-]

You'd also probably have to at least to some degree integrate the idea that it's 'okay' (not correct, just acceptable) to be irrational into your general thought process, to avoid unintentional signaling that you think poorly of them. If anything, irrational people are more likely to notice such subtle signals, since so much of their communication is based on them.

Or, you could just treat the existence of irrationality as a mere fact, like the fact that water freezes or runs downhill. Facts are not a matter of correctness or acceptability, they just are.

In fact (no pun intended), assigning "should-ness" to facts or their opposites in our brains is a significant force in our own irrationality. To say that people "should" be rational is like saying that water "should" run uphill - it says more about your value system than about the thing supposedly being pointed to.

Functionally, beliefs about "should" and "should not" assign aversive consequences to current reality - if I say water "should" run uphill, then I am saying that is is bad that it does not. The practical result of this is to incur an aversive emotional response every time I am exposed to the fact that water runs downhill -- a response which does not benefit me in any way.

A saner, E-prime-like translation of "water should run uphill" might be, "I would prefer that water ran uphill". My preference is just as unlikely to be met in that case, but I do not experience any aversion to the fact that reality does not currently match my preference. And I can still experience a positive emotional response from, say, crafting nice fountains that pump water uphill.

It seems to me that a rationalist would experience better results in life if he or she did not experience aversive emotions from exposure to common facts... such as the fact that human beings run on hardware that's poorly designed for rationality.

Without such aversions, it would be unnecessary to craft complex strategies to avoid signaling them to others. And, equally important, having aversive responses to impersonal facts is a strong driver of motivated reasoning that's hard to detect in ourselves!

Comment author: Vladimir_Nesov 30 May 2009 05:51:26PM *  1 point [-]

Good summary; the confusion of treating natural mindless phenomena with intentional stance was addressed in the Three Fallacies of Teleology post.

When it is possible to change the situation, emotion directed the right way acts as reinforcement signal, and helps to learn the correct behavior (and generally to focus on figuring out a way of improving the situation). Attaching the right amount of right emotions to the right situations is an indispensable tool, good for efficiency and comfort.

Comment author: pjeby 30 May 2009 07:20:13PM 1 point [-]

When it is possible to change the situation, emotion directed the right way acts as reinforcement signal, and helps to learn the correct behavior (and generally to focus on figuring out a way of improving the situation). Attaching the right amount of right emotions to the right situations is an indispensable tool, good for efficiency and comfort.

The piece you may have missed is that even if the situation can be changed, it is still sufficient to use a positive reinforcement to motivate action, and in human beings, it is generally most useful to use positive reinforcement to motivate positive action.

This is because, on the human platform at least, positive reinforcement leads to exploratory, creative, and risk-taking behaviors, whereas negative reinforcement leads to defensive, risk-avoidance, and passive behaviors. So if the best way to change a situation is to avoid it, then by all means, use negative reinforcement.

However, if the best way to change the situation is to engage with it, then negative emotions and "shoulds" are your enemy, not your friend, as they will cause your mind and body to suggest less-useful behaviors (and signals to others).

Comment author: Vladimir_Nesov 30 May 2009 07:30:55PM *  0 points [-]

IAWYC, modulo the use of "should": at least with connotations assumed on Less Wrong, it isn't associated with compulsion or emotional load, it merely denotes preference. "Ought" would be closer.

Comment author: pjeby 30 May 2009 07:54:29PM 4 points [-]

IAWYC, modulo the use of "should": at least with connotations assumed on Less Wrong, it isn't associated with compulsion or emotional load, it merely denotes preference. "Ought" would be closer.

It's true that in technical contexts "should" has less emotional connotation; however even in say, standards documents, one capitalizes SHOULD and MUST to highlight the technical, rather than colloquial sense of these words. Banishing them from one's personal vocabulary greatly reduces suffering, and is the central theme of "The Work" of Byron Katie (who teaches a simple 4-question model for turning "shoulds" into facts and felt-preferences).

Among a community of rationalists striving for better communication, it would be helpful to either taboo the words or create alternatives. As it is, a lot of "shoulds" get thrown around here without reference to what goal or preference the shoulds are supposed to serve.

"One should X" conveys no information about what positive or negative consequences are being asserted to stem from doing or not-doing X -- and that's precisely the sort of information that we would like to have if we are to understand each other.

Comment author: loqi 31 May 2009 12:16:37AM 0 points [-]

Agreed. Even innocuous-looking exceptions, like phrases of the form, "if your goal is to X, then you should Y", have to make not-necessarily-obvious assumptions about what exactly Y is optimizing.

Comment author: Vladimir_Nesov 30 May 2009 08:46:28PM *  0 points [-]

Avoiding existing words is in many cases a counterproductive injunction, it's a normal practice when words get stolen for terms of art. Should refers to a sum total of ideal preference, the top level terminal goal, over all of the details (consequences) together.

Should may require a consequentialist explanation for instrumental actions, or a moral argument for preference over consequences.

Comment author: AdeleneDawner 30 May 2009 08:12:23PM 0 points [-]

Agreed. This is one of the major themes of some (most?) meditation practices and seems to be one of the most useful.

Comment author: AdeleneDawner 30 May 2009 03:50:27PM *  0 points [-]

That's just what I was trying to get at. Thanks for the clarification.