= 27d567bc2d48d9f40e9ccc20ea370b15)
Annoyance comments on A social norm against unjustified opinions? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (158)
It took me years to even recognize that I was doing that, and I still haven't managed to stop completely.
One obstacle: as long as they aren't expected to produce obvious results to meet your expectations, people really, really like being given too much credit. And they really, really dislike being given precisely enough credit when they're nothing special, even if it lets them off the hook.
Many of my social 'problems' began once I recognized that other people didn't think like I did, and were usually profoundly stupid. That's not a recognition that lends itself to frictionless interaction with others.
This little tidbit highlights so much of what's wrong with this community:
"Many of my social 'problems' began once I recognized that other people didn't think like I did, and were usually profoundly stupid. That's not a recognition that lends itself to frictionless interaction with others."
You'd think a specimen of your gargantuan brainpower would have the social intelligence to handily conceal your disdain for the commonfolk. Perhaps it's some sort of signaling?
I think you're underestimating the degree of social intelligence required. To pull that off while still keeping the rationalistic habits that such people find offensive, you'd have to:
You'd also probably have to at least to some degree integrate the idea that it's 'okay' (not correct, just acceptable) to be irrational into your general thought process, to avoid unintentional signaling that you think poorly of them. If anything, irrational people are more likely to notice such subtle signals, since so much of their communication is based on them.
Or, you could just treat the existence of irrationality as a mere fact, like the fact that water freezes or runs downhill. Facts are not a matter of correctness or acceptability, they just are.
In fact (no pun intended), assigning "should-ness" to facts or their opposites in our brains is a significant force in our own irrationality. To say that people "should" be rational is like saying that water "should" run uphill - it says more about your value system than about the thing supposedly being pointed to.
Functionally, beliefs about "should" and "should not" assign aversive consequences to current reality - if I say water "should" run uphill, then I am saying that is is bad that it does not. The practical result of this is to incur an aversive emotional response every time I am exposed to the fact that water runs downhill -- a response which does not benefit me in any way.
A saner, E-prime-like translation of "water should run uphill" might be, "I would prefer that water ran uphill". My preference is just as unlikely to be met in that case, but I do not experience any aversion to the fact that reality does not currently match my preference. And I can still experience a positive emotional response from, say, crafting nice fountains that pump water uphill.
It seems to me that a rationalist would experience better results in life if he or she did not experience aversive emotions from exposure to common facts... such as the fact that human beings run on hardware that's poorly designed for rationality.
Without such aversions, it would be unnecessary to craft complex strategies to avoid signaling them to others. And, equally important, having aversive responses to impersonal facts is a strong driver of motivated reasoning that's hard to detect in ourselves!
Good summary; the confusion of treating natural mindless phenomena with intentional stance was addressed in the Three Fallacies of Teleology post.
When it is possible to change the situation, emotion directed the right way acts as reinforcement signal, and helps to learn the correct behavior (and generally to focus on figuring out a way of improving the situation). Attaching the right amount of right emotions to the right situations is an indispensable tool, good for efficiency and comfort.
The piece you may have missed is that even if the situation can be changed, it is still sufficient to use a positive reinforcement to motivate action, and in human beings, it is generally most useful to use positive reinforcement to motivate positive action.
This is because, on the human platform at least, positive reinforcement leads to exploratory, creative, and risk-taking behaviors, whereas negative reinforcement leads to defensive, risk-avoidance, and passive behaviors. So if the best way to change a situation is to avoid it, then by all means, use negative reinforcement.
However, if the best way to change the situation is to engage with it, then negative emotions and "shoulds" are your enemy, not your friend, as they will cause your mind and body to suggest less-useful behaviors (and signals to others).
IAWYC, modulo the use of "should": at least with connotations assumed on Less Wrong, it isn't associated with compulsion or emotional load, it merely denotes preference. "Ought" would be closer.
It's true that in technical contexts "should" has less emotional connotation; however even in say, standards documents, one capitalizes SHOULD and MUST to highlight the technical, rather than colloquial sense of these words. Banishing them from one's personal vocabulary greatly reduces suffering, and is the central theme of "The Work" of Byron Katie (who teaches a simple 4-question model for turning "shoulds" into facts and felt-preferences).
Among a community of rationalists striving for better communication, it would be helpful to either taboo the words or create alternatives. As it is, a lot of "shoulds" get thrown around here without reference to what goal or preference the shoulds are supposed to serve.
"One should X" conveys no information about what positive or negative consequences are being asserted to stem from doing or not-doing X -- and that's precisely the sort of information that we would like to have if we are to understand each other.
Agreed. Even innocuous-looking exceptions, like phrases of the form, "if your goal is to X, then you should Y", have to make not-necessarily-obvious assumptions about what exactly Y is optimizing.
Avoiding existing words is in many cases a counterproductive injunction, it's a normal practice when words get stolen for terms of art. Should refers to a sum total of ideal preference, the top level terminal goal, over all of the details (consequences) together.
Should may require a consequentialist explanation for instrumental actions, or a moral argument for preference over consequences.
Agreed. This is one of the major themes of some (most?) meditation practices and seems to be one of the most useful.
That's just what I was trying to get at. Thanks for the clarification.
The problems you cite in bullets are only nontrivial if you don't sufficiently value social cohesion. My biggest faux pas have sufficiently conditioned me to make them less often because I put a high premium on that cohesion. So I think it's less a question of social intelligence and more one of priorities. I don't have to keep "constant focus" - after a few faux pas it becomes plainly apparent which subjects are controversial and which aren't, and when we do come around to touchy ones I watch myself a little more.
I thought I would get away with that simplification. Heh.
Those skills do come naturally to some people, but not everyone. They certainly don't come naturally to me. Even if I'm in a social group with rules that allow me to notice that a faux pas has occurred (not all do; some groups consider it normal to obscure such things to the point where I'll find out weeks or months later, if at all), it's still not usually obvious what I did wrong or what else I could do instead, and I have to intentionally sit down and come up with theories that I may or may not even have a chance to test.
Right, I get that people fare differently when it comes to this stuff, but I do think it's a matter of practice and attention more than innate ability (for most people). And this is really my point, that the sort of monastic rationality frequently espoused on these boards can have politically antirational effects. It's way easier to influence others if you first establish a decent rapport with them.
I don't at all disagree that the skills are good to learn, especially if you're going to be focusing on tasks that involve dealing with non-rationalists. I think it may be a bit of an over generalization to say that they should be a high priority for everyone, but probably not much of one.
I do have a problem with judging people for not having already mastered those skills, or for having higher priorities than tackling those skills immediately with all their energy, though, which seems to be what you're doing. Am I inferring too much when I come to that conclusion?
Look, this whole thread started because of Annoyance's judgment of people who have higher priorities than rationality, right? Did you have a problem with that?
All I'm saying is that this community in general gives way too short shrift to the utility of social cohesion. Sorry if that bothers you.
Quote, please?
Most of what he said condenses to "people who are not practicing rationality are irrational", which is only an insult if you consider 'irrational' to be an insult, which I didn't see any evidence of. I saw frustration at the difficulty in dealing with them without social awkwardness, but that's not the same.
Have I missed something?
Yes, and most of what I said reduces to "Annoyance is not practicing rationality with statements like "'social cohesion is one of the enemies of rationality.'" You said you had a "problem" with my contention and then I pointed out that Annoyance had made a qualitatively similar claim that hadn't bothered you. Aside from our apparent disagreement on the point I don't get how my claim could be a problem for you.
I think I've made myself clear and this is getting tiresome so I'll invite you to have the last word.
Social cohesion is one of the enemies of rationality.
It's not necessarily so in that it's not always opposed to it, but it is incompatible with the mechanisms that bring it about and permit it to error-correct. It tends to reinforce error. When it happens to reinforce correctness, it's not needed, and when it doesn't, it makes it significantly harder to correct the errors.
"When it happens to reinforce correctness, it's not needed"
Can you elaborate?
I'll note that rationality isn't an end. My ideal world state would involve a healthy serving of both rationality and social cohesion. There are many situations in which these forces work in tandem and many where they're at odds.
A perfect example is this site. There are rules the community follows to maintain a certain level of social cohesion, which in turn aides us in the pursuit of rationality. Or are the rules not needed?
Why can't it be?
How is that demonstrated?
Those rules are rarely discussed outright, at least not comprehensively.
I'm pretty sure if I started posting half of my comments in pig-Latin or French or something, for no apparent reason, and refused to explain or stop, I'd be asked to leave fairly quickly, though. That all communication will be in plain English unless there's a reason for it not to be is one example. I'm sure there are others.
It's demonstrated by the fact that you can up/down vote and report anyone's posts, and that you need a certain number of upvotes to write articles. This is a method of policing the discourse on the site so that social cohesion doesn't break down to an extent which impairs our discussion. These mechanisms "reinforce correctness," in your terms. So I'll ask again, can we do away with them?
I don't think humanity follows obviously from rationality, which is what I meant about rationality being a means rather than an end.
You're assuming a fact not in evidence.
So you tell me what you think they're for, then.
I disagree. It is rational to exploit interpersonal communication for clarity between persons and comfortable use. If the 'language of rationality' can't be understood by the 'irrational people', it is rational to translate best you can, and that can include utilizing societal norms. (For clarity and lubrication of the general process.)
Yes, I agree - my point was that the skill of translating is a difficult one to acquire, not that it's irrational to acquire it.
Oh, I'm sorry I misunderstood you. Yeah, it can be tiring. I'm a fairly introverted person and need a good amount of downtime between socialization. I guess I was projecting a little -- I use to think social norms were garbage and useless, until I realized neglecting their utility was irrational and it was primarily an emotional bias against them in never feeling like I 'fit in'. Sometimes it feels like you never stop discovering unfortunate things about yourself...
I agree here: Reading stuff like this totally makes me cringe. I don't know why people of above average intelligence want to make everyone else feel like useless proles, but it seems pretty rampant. Some humility is probably a blessing here, I mean, as frustrating as it is to deal with the 'profoundly stupid', at least you yourself aren't profoundly stupid.
Of course, they probably think given the same start the 'profoundly stupid' person was given, they would have made the best of it and would be just as much of a genius as they are currently.
It's a difficult realization, when you become aware you're more intelligent then average, to be dropped into the pool with a lot of other smart people and realize you really aren't that special. I mean, in a world of some six billion odd, if you are a one-in-a-million genius, that still means you likely aren't in the top hundred smartest people in the world and probably not in the top thousand. It kind of reminds me of grad school stories I've read, with kids who think they are going to be a total gift to their chosen subject ending up extremely cynical and disappointed.
I think people online like to exaggerate their eccentricity and disregard for societal norms in an effort to appeal to the stereotypes for geniuses. I've met a few real geniuses IRL and I know you can be a genius without being horribly dysfunctional.
Rationality and intelligence are not the same thing - I've seen plenty of discussions here despairing about the existence of obviously-intelligent people, masters in their fields, who haven't decided to practice rationality. I also know people who are observably less intelligent than I am, who practice rationality about as well as I do. One major difference between people in that latter group, and people who are not practicing rationality, no matter what the irrational peoples' intelligence levels are, is that those people don't get offended when someone points out a flaw in their reasoning, just as I don't get offended when they, or even people who are not practicing rationality, point out a flaw in mine. People who are less intelligent will probably progress more slowly with rationality, as with any mental skill-set, but that's not under discussion here. The irrational unwillingness to accept criticism is.
Being called 'profoundly stupid' is not exactly a criticism of someone's reasoning. (Not that anybody was called that.) I think we're objecting to this because of how it'll offend people outside of the 'in group' anyway. Besides that, As much as we might wish we were immune to the emotional shock or glee at our thoughts and concepts being ridiculed or praised. I think it would be a rarity here to find someone who didn't. People socializing and exchanging ideas is a type of system -- It has to be understood and used effectively in order to produce the best results -- and calling, essentially, everybody who disagrees with you 'profoundly stupid' is not good social lubrication.
You appear to be putting words into my mouth, but I'm currently too irritated to detangle this much beyond that point.
"Giving people too much credit" was a reference to peoples' desire to be rational. I tend to assume that that's significantly above zero in every case, even though the evidence does not seem to support that assumption. This is a failure to be rational on my part. (I doubt I'll fix that; it's the basis for most of my faith in humanity.)
I make no such assumption about intelligence (I do not assume that people want to be more intelligent than they are), and make a conscious effort to remove irrational biases toward intelligent people from my thought process when I encounter them. I have been doing so for years, with a significant degree of success, especially considering that I was significantly prejudiced against less intelligent people, before I realized that it was wrong to hold that view.
I have also put significant effort into learning how to bridge both of those communication gaps, and the skills required in each case are different. When I'm simply dealing with someone who's less intelligent, I moderate my vocabulary, use lots of supporting social signaling, make smaller leaps of logic, and request feedback frequently to make sure I haven't lost them. (Those skills are just as useful in regular conversation as they are in explaining things.) When I'm dealing with someone who's not practicing rationality, I have to be very aware of their particular worldview, and only thoughtfully challenge it - which requires lots of complicated forethought, and can require outright lies.
The lack of either of those sets of communication skills will make dealing with the relevant people difficult, and can lead to them thinking poorly of you, whether you actually are prejudiced against them or not. Assuming that someone who does not have one of those sets of skills is prejudiced does not, in practice, work - there's a very high risk of getting a false-positive.
I wrote this rant before I saw the thing above. I'm not deleting it, because someone may find this useful, but the issue has been resolved. :)
You should expand this into a top-level post. Communication is difficult and I think most people could use advice about it. As it stands, it sounds like broad strokes which are obviously good ideas, but probably hard to implement without more details.
I've been considering it, actually, for my own use if not to post here. I think it'd be useful in several ways to try to come up with actual wordings for the tricks I've picked up.
A person who is 'thinking' irrationally can only be challeneged to the degree that they're being rational. If they eschew rationality completely, there isn't any way to communicate with them.
What have you actually accomplished, if you use social signals to get someone to switch their concept-allegiances?
I thought we'd already defined "practicing rationality" as "intentionally trying to make rational decisions and intentionally trying to become more rational". Whether we had or not, that was what I meant by the term.
Someone can be being somewhat rational without 'practicing' rationality, and to the degree that they can accurately predict what effects follow what causes, or accomplish other tasks that depend on rationality, every person I know is at least somewhat rational. Even animals can be slightly rational - cats for example are well known for learning that the sound of a can opener is an accurate sign that they may be fed in the near future, even if they aren't rational enough to make stronger predictions about which instances of that sound signal mealtime.
While social signaling can be used on its own to cause someone to switch their allegiances to concepts that they don't value especially highly, that's not the only possible use of it, and it's not a use I consider acceptable. The use of social-signaling that I recommend is intended to keep a person from becoming defensive while 'rationality-level appropriate' rational arguments are used to actually encourage them to change their mind.
No, only if you rationally try to make rational decisions and rationally try to become more rational.
If you're acting irrationally, you're not practicing rationality, in the same way that you're not practicing vegetarianism if you're eating meat.
Isn't it obvious? Almost everyone is a "useless prole", as you put it, and even the people who aren't have to sweat blood to avoid that fate.
Recognizing that unpleasant truth is the first step towards becoming non-useless - but most people can't think usefully enough to recognize it in the first place, so the problem perpetuates itself.
I know I'm usually a moron. I've also developed the ability to distinguish quality thinking from moronicity, which makes it possible for me to (slowly, terribly slowly) wean myself away from stupid thinking and reinforce what little quality I can produce. That's what makes it possible for me to occasionally NOT be a moron, at least at a rate greater than chance alone would permit.
It's the vast numbers of morons who believe they're smart, reasonable, worthwhile people that are the problem.
I was reading around on the site today, and I think I've figured out why this attitude sends me running the other way. What clued me in was Eliezer's description of Spock in his post "Why Truth? And...".
Eliezer's point there is that Spock's behavior goes against the actual ideals of rationality, so people who actually value rationality won't mimic him. (He's well enough known that people who want to signal that they're rational will likely mimic him, and people who want to both be and signal being rational will probably mimic him in at least some ways, and also note that the fact that reversed stupidity is not intelligence is relevant.)
It may come as a shock, but in my case, being rational is not my highest priority. I haven't actually come up with a proper wording for my highest priority yet, but one of my major goals in pursuing that priority is to facilitate a universal ability for people to pursue their own goals (with the normal caveats about not harming or overly interfering with other people, of course). One of the primary reasons I pursue rationality is to support that goal. I suspect that this is not an uncommon kind of reason for pursuing rationality, even here.
As I mentioned in the comment that I referenced, I've avoided facing the fact that most people prefer not to pursue rationality, because it appears that that realization leads directly to the attitude you're showing here, and I can reasonably predict that if I were to have the attitude you're showing here, I would no longer support the idea that everyone should have as much freedom as can be arranged, and I don't want to do that. Very few people would want to take the pill that'd turn them into a psychopath, even if they'd be perfectly okay with being a psychopath after they took the pill.
But there's an assumption going on in there. Does accepting that fact actually have to lead to that attitude? Is it impossible to be an x-rationalist and still value people?
This is something I’ve thought a lot about. I’m worried about the consequences of certain negative ideologies present here on Less Wrong, but, actually, I feel that x-rationality, combined with greater self-awareness, would be the best weapon against them. X-rationality -- identifying facts that are true and strategies that work -- is inherently neutral. The way you interpret those facts (and what you use your strategies for) is the result of your other values.
Consider, to begin with, the tautology that 99.7% of the population is less intelligent than 0.3% of the population, by some well-defined, arbitrary metric of intelligence. Suppose also, that someone determined they were in the top 0.3%. They could feel any number of ways about this fact: completely neutral, for example, or loftily superior, or weightily responsible. Seen in this way, feeling contempt for "less intelligent" people is clearly the result of a worldview biased in some negative way.
Generally, humanity is so complex that however anyone feels about humanity says more about them than it does about humanity. Various forces (skepticism and despair; humanism and a sense of purpose) have been vying throughout history: rationality isn’t going to settle it now. We need to pick our side and move on … and notice which sides other people have picked when we evaluate their POV.
I always find it ironic, when 'rationalists' are especially misanthropic here on Less Wrong, that Eliezer wants to develop a friendly AI. Implicit with this goal -- built right in -- is the awareness that rationality alone would not induce the machine to be friendly. So why would we expect that a single-minded pursuit of rationality would not leave us vulnerable to misanthropic forces? Just as we would build friendliness into a perfectly logical, intelligent machine; we must build friendliness into our ideology before we let go of “intuition” and other irrational ways we have of “feeling” what is right, because they contain our humanism, which is outside rationality.
We do not want to be completely rational because being rational is neutral. Being more neutral without perfect rationality would leave us vulnerable to negative forces, and, anyway, we want to be a positive force.
If we assume he has goals other than simply being a self-abasing misanthrope, the attitude Annoyance is showing is far from rational. Arbitrarily defining the vast majority of humans as useless "problems" is, ironically, itself a useless and problematic belief, and it represents an even more fundamental failure than being Spocklike -- Spock, at least, does not repeatedly shoot himself in the foot and then seek to blame anything but himself.
I've pretty much figured that out. If nothing else, Annoyance is being an excellent example of that right now.
Next question: Is it something about this method of approaching rationality that encourages that failure mode? How did Annoyance fall off the path, and can I avoid doing the same if I proceed?
I'm starting to think that the answer to that last question is yes, though.
While I find conversations with Annoyance rather void, I would encourage you to not try and lift (him ?) up as an example of falling off the path or entering failure modes. If you care about the question I would make a post using generic examples. This does a few things:
That's very good advice. However, I'm not going to take it today, and probably won't at all. It seems more useful at this point to take a break from this entirely and give myself a chance to sort out the information I've already gained.
I'll definitely be interested in looking at it, in a few days, if someone else wants to come up with that example and continue thinking about it here.
I would agree.
I pass. The discussion of that topic would be interesting to me but writing the article is not. I have too many partial articles as it is... :P
Annoyance, you're still dodging the question. Joe didn't ask whether or not in your opinion everyone is a useless prole, he asked why it's useful to make people feel like that. Your notion that "social cohesion is the enemy of rationality" was best debunked, I think by pjeby's point here:
http://lesswrong.com/lw/za/a_social_norm_against_unjustified_opinions/rrk
more flies with honey and all that.
I don't want to catch flies.
Annoyance, your argument has devolved into inanity. If you don't want to popularly cultivate rationality then you disagree with one of the core tenets of this community. It's in the second paragraph of the "about" page:
"Less Wrong is devoted to refining the art of human rationality - the art of thinking. The new math and science deserves to be applied to our daily lives, and heard in our public voices."
Your circular word games do no good for this community.
Or perhaps simply the recognition that it's sometimes impossible to fluff other people's egos and drive discussion along rational paths at the same time.
If people become offended when you point out weaknesses in their arguments - if they become offended if you even examine them and don't automatically treat their ideas as inherently beyond reproach - there's no way to avoid offending them while also acting rationally. It becomes necessary to choose.
Really? Have you tried, maybe, just not pointing out the weaknesses in their arguments? Mightn't that be the rational thing to do? Just a polite smile and nod, or a gentle, "Have you considered some alternative?" Or even, "You may well be right." (This is true of pretty much any non-contradictory statement.) Or there are many different ways to argue with someone without being confrontational. Asking curious-sounding questions works fairly well.
It's generally easy to recognize how well a person will react to an argument against him. If you have basic people skills, you'll be able to understand what type of argument/approach will communicate your point effectively, and when you simply don't have a chance. The idea that it's necessary to offend people to act rationally seems completely absurd (at least in this context). If it's going to offend them, it's going to accomplish the opposite of your goal, so, rationally, you shouldn't do it.
This whole discussion reminds me of the Dave Barry quote that may well have been used earlier on this site:
"I argue very well. Ask any of my remaining friends. I can win an argument on any topic, against any opponent. People know this, and steer clear of me at parties. Often, as a sign of their great respect, they don't even invite me."
This. Is. Not. Winning.
I was going to say "there are more workarounds than you think", but that's probably my selection bias talking again. That said, there are workarounds, in some situations. It's still not a trivial thing to learn, though.
It's not just nontrivial, it's incredibly hard. Engaging "system 2" reasoning takes a lot of effort, lowering sensitivity to, and acute awareness of, social cues and signals.
The mindset of "let's analyze arguments to find weaknesses," aka Annoynance's "rational paths," is a completely different ballgame than most people are willing to play. Rationalists may opt for that game, but they can't win, and may be reinforcing illogical behavior. Such a rationalist is focused on whether arguments about a particular topic are valid and sound, not the other person's rational development. If the topic is a belief, attempting to reason it out with the person is counterproductive. Making no ground when engaging with people on a topic should be a red flag: "maybe I'm doing the wrong thing."
Does anyone care enough for me to make a post about workarounds? Maybe we can collaborate somehow Adelene, I have a little experience in this area.
Engaging system 2 is precisely what you don't want to do, since evolutionarily speaking, a big function of system 2 is to function as a decoy/shield mechanism for keeping ideas out of a person. And increasing a person's skill at system 2 reasoning just increases their resistance to ideas.
To actually change attitudes and beliefs requires the engagement of system 1. Otherwise, even if you convince someone that something is logical, they'll stick with their emotional belief and just avoid you so they don't have to deal with the cognitive dissonance.
(Note that this principle also applies to changing your own beliefs and attitudes - it's not your logical mind that needs convincing. See Eliezer's story about overcoming a fear of lurking serial killers for an example of mapping System 2 thinking to System 1 thinking to change an emotional-level belief.)
pjeby, sorry I wasn't clear, I should have given some context. I am referencing system 1 and 2 as simplified categories of thinking as used by cognitive science, particularly in behavioral economics. Here's Daniel Kahneman discussing them. I'm not sure what you're referring to with decoys and shields, which I'll just leave at that.
To add to my quoted statement, workarounds are incredibly hard, and focusing on reasoning (system 2) about an issue or belief leaves few cycles for receiving and sending social cues and signals. While reasoning, we can pick up those cues and signals, but they'll break our concentration, so we tend to ignore them while reasoning carefully. The automatic, intuitive processing of the face interferes with the reasoning task; e.g. we usually look somewhere else when reasoning during a conversation. To execute a workaround strategy, however, we need to be attuned to the other person.
When I refer to belief, I'm not referring to fear of the dark or serial killers, or phobias. Those tend to be conditioned responses--the person knows the belief is irrational--and they can be treated easily enough with systematic desensitization and a little CBT thrown in for good measure. Calling them beliefs isn't wrong, but since the person usually knows they're irrational, they're outside my intended scope of discussion: beliefs that are perceived by the believer to be rational.
People are automatically resistant to being asked to question their beliefs. Usually it's perceived as unfair, if not an actual attack on them as a person: those beliefs are associated with their identity, which they won't abandon outright. We shouldn't expect them to. It's unrealistic.
What should we do, then? Play at the periphery of belief. To reformulate the interaction as a parable: We'll always lose if we act like the wind, trying to blow the cloak off the traveller. If we act like the sun, the traveller might remove his cloak on his own. I'll think about putting a post together on this.
My hypothesis is that reasoning as we know it evolved as a mechanism to both persuade others, and to defend against being persuaded by others.
Consider priming, which works as long as you're not aware of it and therefore defending against it. But it makes no sense to evolve a mechanism to avoid being primed, unless the priming mechanism were being exploited by our tribe-mates. (After all, they're the only ones besides us with the language skill to trigger it.)
In other words, once we evolved language, we became more gullible, because we were now verbally suggestible. This would then have resulted in an arms race of intelligence to both persuade, and defend against persuasion, with tribal status and resources as the prize.
And once we evolved to the point of being able to defend ourselves against any belief-change we're determined to avoid, the prize would've become being able to convince neutral bystanders who didn't already have something at stake.
The system 1/2 distinctions cataloged by Stanovich & West don't quite match my own observation, in that I consider any abstract processing to be system 2, whether it's good reasoning or fallacious, and whether it's cached or a work-in-progress. (Cached S2 reasoning isn't demanding of brainpower, and in fact can be easily parroted back in many forms once an appropriate argument has been heard, without the user ever needing to figure it out for themselves.)
In my view, the primary functional purpose of human of reasoning is to persuade or prevent persuasion, with other uses being an extra bonus. So in this view, using system 2 for truly rational thought is actually an abuse of the system... which would explain why it's so demanding of cognitive capacity, compared to using it as a generator of confabulation and rhetoric. And it also explains why it requires so much learning to use properly: it's not what the hardware was put there for.
The S&W model is IMO a bit biased by the desire to find "normative" reasoning (i.e., correct reasoning) in the brain, even though there's really no evolutionary reason for us to have truly rational thought or to be particularly open-minded. In fact, there's every evolutionary reason for us to not be persuadable whenever we have something at stake, and to not reason things out in a truly fair or logical manner.
Hence, some of the attributes they give system 2 are (in my view) attributes of learned reasoning running on top of system 2 in real time, rather than native attributes of system 2 itself, or reflective of cached system 2 thinking.
Anyway, IAWYC re: the rest, I just wanted to clarify this particular bit.
Actually, system one can handle a surprising amount of abstraction; I don't have one handy, but any comprehensive description of conceptual synesthesia should do a good job of explaining it. (I'm significantly enough conceptually synesthetic that I don't need it explained, and have never actually needed an especially good reference before.)
The fact that I can literally see that the concept 'deserve X' depends on the emotional version of the concept 'should do X', because the pattern for one contains the pattern for the other, makes it very clear to me that such abstractions are not dependent on the rational processing system.
It's also noteworthy that synesthesia appears to be a normal developmental phase; it seems pretty likely to me that I'm merely more aware of how my brain is processing things, rather than having a radically different mode of processing altogether.
I'd certainly be interested in that. My own definitions are aimed at teaching people not to abstract away from experience, including emotional experience. Certainly there is some abstraction at that level, it's just a different kind of abstraction (ISTM) than system 2 abstraction.
In particular, what I'm calilng system 1 does not generally use complex sentence structure or long utterances, and the referents of its "sentences" are almost always concrete nouns, with its principal abstractions being emotional labels rather than conceptual ones.
I consider "should X" and "deserve X" to both be emotional labels, since they code for attitude and action towards X, and so both are well within system 1 scope. When used by system 2, they may carry totally different connotations, and have nothing to do with what the speaker actually believes they deserve or should do, and especially little to do with what they'll actually do.
For example, a statement like, "People should respect the rights of others and let them have what they deserve" is absolutely System 2, whereas, a statement like "I don't deserve it" (especially if experienced emotionally) is well within System 1 territory.
It's entirely possible that my definition of system 1/2 is more than a little out of whack with yours or the original S&W definition, but under my definition it's pretty easy to learn to distinguish S1 utterances from S2 utterances, at least within the context of mind hacking, where I or someone else is trying to find out what's really going on in System 1 in relation to a topic, and distinguish it from System 2's confabulated theories.
However, since you claim to be able to observe system 1 directly, this would seem to put you in a privileged position with respect to changing yourself - in principle you should be able to observe what beliefs create any undesired behaviors or emotional responses. Since that's the hard part of mind hacking IME, I'm a bit surprised you haven't done more with the "easy" part (i.e. changing the contents of System 1).
I feel I should jump in here, as you appear to be talking past each other. There is no confusion in the system 1/system 2 distinction; you're both using the same definition, but the bit about decoys and shields was actually the core of PJ's post, and of the difference between your positions. PJ holds that to change someone's mind you must focus on their S1 response, because if they engage S2, it will just rationalize and confabulate to defend whatever position their S1 holds. Now, I have no idea how one would go about altering the S1 response of someone who didn't want their response altered, but I do know that many people respond very badly to rational arguments that go against their intuition, increasing their own irrationality as much as necessary to avoid admitting their mistake.
I don't believe we are, because I know of no evidence of the following:
Perhaps one or both of us misunderstands the model. Here is a better description of the two.
Originally, I was making a case that attempting to reason was the wrong strategy. Given your interpretation, it looks like pjeby didn't understand I was suggesting that, and then suggested essentially the same thing.
My experience, across various believers (Christian, Jehovah's Witness, New Age woo-de-doo) is that system 2 is never engaged on the defensive, and the sort of rationalization we're talking about never uses it. Instead, they construct and explain rationalizations that are narratives. I claim this largely because I observed how "disruptable" they were during explanations--not very.
How to approach changing belief: avoid resistance by avoiding the issue and finding something at the periphery of belief. Assist in developing rational thinking where the person has no resistance, and empower them. Strategically, them admitting their mistake is not the goal. It's not even in the same ballpark. The goal is rational empowerment.
Part of the problem, which I know has been mentioned here before, is unfamiliarity with fallacies and what they imply. When we recognize fallacies, most of the time it's intuitive. We recognize a pattern likely to be a fallacy, and respond. We've built up that skill in our toolbox, but it's still intuitive, like a chess master who can walk by a board and say "white mates in three."
This. Exactly this. YES.
Tell them stories. If you'll notice, that's what Eliezer does. Even his posts that don't use fiction per se use engaging examples with sensory detail. That's the stuff S1 runs on.
Eliezer uses a bit more S2 logic in his stories than is perhaps ideal for a general audience; it's about right for a sympathetic audience with some S2+ skills, though.
On a general audience, what might be called "trance logic" or "dramatic logic" works just fine on its own. The key is that even if your argument can be supported by S2 logic, to really convince someone you must get a translation to S1 logic.
A person who's being "reasonable" may or may not do the S2->S1 translation for you. A person who's being "unreasonable" will not do it for you; you have to embed S1 logic in the story so that any effort to escape it with S2 will be unconvincing by comparison.
This, by the way, is how people who promote things like intelligent design work: they set up analogies and metaphors that are much more concretely convincing on the S1 level, so that the only way to refute them is to use a massive burst of S2 reasoning that leaves the audience utterly unconvinced, because the "proof" is sitting right there in S1 without any effort being required to accept it.
I hadn't actually found the system 1/system 2 meme before this, but it maps nicely onto how I handle those situations. The main trick is to make lots of little leaps of logic, instead of one big one, while pushing as few emotional buttons as you can get away with, and using the emotional buttons you do push to guide the conversation along.
An example of that is here. In the original example, telling someone directly that they're wrong pushes all kinds of emotional buttons, and a fully thought out explanation of why is obviously too much for them to handle with system one, so it's going to fall flat, unless they want to understand why they're wrong, which you've already interfered with by pushing their buttons.
In my example, I made a much smaller leap of logic - "you're using a different definition of 'okay' than most people do" - which can be parsed by system one, I think. I also used social signaling rather than words to communicate that the definition is not okay, which is a good idea because social signaling can communicate that with much more finesse and fewer emotional buttons pushed, and because people are simply wired to go along with that kind of influence more easily.
No kidding.
My sanity-saver ... but obviously not rationality-saver... has been to learn to encourage the people I'm dealing with to be more rational, at least when dealing with me. My inner circle of friends is made up almost entirely of people who ask themselves and each other that kind of question just as a matter of course, now, and dissect the answers to make sure they're correct and rational and well-integrated with the other things we know about each other.
That doesn't help at all when I'm trying to think about society in general, though.