For example, when discussing gender-related problems, it seems inevitable that some proposed solutions will generally be better for men, and other solutions will generally be better for women.
Even this is too simplistic. Polygyny is better for high status men and most women, and bad for low status men and high status women. (And this is before you get to the evolutionary effects, which are both positive and negative and thus hard to judge!)
I meant that to be a simple example of how social policies/norms tend to create winners and losers relative to other policies or the status quo, and didn't mean to suggest that the winners and losers of gender-related policies will all or mostly divide along gender lines. I've reworded the sentence to make it clearer. Thanks.
I think people overrate the prevalence of fundamental value difference as a way of protecting a bad map of human value.
I don't think we are anywhere near the point where fundamental value differences between men and women (or any other groups) are relevant.
Nobody wants to be oppressed, nobody wants to die, nobody wants to be hurt or sick, everybody wants more good friends, everybody wants more love, everybody wants to be more autonomous, etc etc.
To me, the point where fundamental value differences matter is when you've solved all of that and the biggest thing left to argue about is whose aesthetic sense the architecture should be optimized for.
Suppose Alice doesn't want Alice to die, Bob doesn't want Bob to die, and these are the only people and values in the world. Do you think these are not "different" values? (Note that I explicitly mentioned selfish values in the OP as an example of what I meant by "different values".) More importantly, wouldn't such values lead to the necessity of bargaining over how to solve problems that affect both of them?
This kind of situation is usually called "conflict of interest". I think using "value differences" is confusing terminology, at least to me it suggests some more fundamental difference such as sacredness vs avoiding harm.
Ah, that makes sense. (I was wondering why nyan_sandwich's comment was being upvoted so much when I already mentioned selfish values in the OP.) To be clear, I'm using "value differences" to mean both selfish-but-symmetric values and "more fundamental difference such as sacredness vs avoiding harm". (ETA: It makes sense to me because I tend to think of values in terms of utility functions that take world states as inputs.) I guess we could argue about which kind of difference is more important but that doesn't seem relevant to the point I wanted to make.
It seems like a relevant distinction in the FAI/CEV theory context, and indirectly relevant in the gender conflicts question. That is, it isn't first-order relevant in the latter case, but seems likely to become so in a thread that is attempting to go meta. Like, say, this one.
good point on selfishness.
What I was getting at is that humans have mostly symmetric values such that they should not disagree over what type of society they want to live in, if they don't get to choose the good end of the stick.
Even if people have symmetric values, the relevant facts are not symmetric. For example everyone values things that money can buy, but some people have much higher abilities to earn money in a free market economy, so there will be conflict over how much market competition to allow or what kind of redistributive policies to have.
if they don't get to choose the good end of the stick
I'm not sure what you mean by this. Are you saying something like, "if they were under a Rawlsian veil of ignorance"? But we are in fact not under a Rawlsian veil of ignorance, and any conclusions we make of the form "If I were under a Rawlsian veil of ignorance, I would prefer society to be organized thus: ..." are likely to be biased by the knowledge of our actual circumstances.
What I was getting at is that humans have mostly symmetric values such that they should not disagree over what type of society they want to live in, if they don't get to choose the good end of the stick.
This seems wrong, except for extremely weak definitions of "mostly". People should definitely disagree about what type of society they want to live in, just a whole lot less than if they were disagreeing with something non-human.
Nobody wants to be oppressed, nobody wants to die, nobody wants to be hurt or sick, everybody wants more good friends, everybody wants more love, everybody wants to be more autonomous, etc etc.
Some people want to oppress, some people want to kill, some people want to hurt others, everybody wants to take status from others for themselves, everyone wants others to be hated, everyone wants others to be subservient to them, ect ect.
(Note: I reflected nyan's assertions to make the point that there's conflict in values; I am not supporting any of the assertions.)
Conflicting terminal values are very much possible. I don't think they exist to a relevant degree among humans.
Conflicting learned values do exist (just look at radical islam for example). I don't think those differences would hold up under reflective value extrapolation.
Selfishness exists and would hold up under value extrapolation. However, that simple value difference is mostly symmetrical, and does not warrant cutting up humanity into groups of people with differing aggregate values.
This isn't negotiating with babyeaters, it's plain old economics of cooperation among humans.
If we're talking about humans, I'm not sure that the distinction between terminal and learned values is very meaningful.
Conflicting terminal values are very much possible. I don't think they exist to a relevant degree among humans.
What exactly do you think is happening in disputes about legal interpretation or legal change?
I'm not saying that every such dispute is caused by value differences, but a substantial number are - and there is a strong social taboo against articulating disputes in the language of value conflict. The socially preferred method of argument is to assert that "common values" support the result that one prefers.
Conflicting terminal values are very much possible. I don't think they exist to a relevant degree among humans.
Why?
Conflicting learned values do exist (just look at radical islam for example). I don't think those differences would hold up under reflective value extrapolation.
Why?
However, that simple value difference is mostly symmetrical, and does not warrant cutting up humanity into groups of people with differing aggregate values.
Assuming a community of people are operating with extrapolated reflective values and yet are still selfish, why then is bargaining not optimal for resolving differences in values (they have values that apply to themselves, and values that apply to others, and so selfishness presumably would make them value the former more than the latter)?
That argument doesn't address the problem of "I want to oppress you", "you want to oppress me".
Extortionate strategies in the Prisoner's Dilemma don't create value as well as nice strategies do, nor do they do as well against one another as nice strategies do; but they beat nice strategies individually.
Some sorts of oppression seem to follow the pattern of extortionate strategies — "I will take advantage of you, and will make it so that you are better off if you let me do so, than if you fight back."
(Real-world examples are probably unhelpful here; I expect that everyone can think of one or two.)
What if the problem is "I want to oppress you, but I know individually being nicer would get me more of what I want, so instead I'm going to recruit allies that will help me oppress you because I think that will get me even more of what I want."
You think conflicting terminal values don't exist to a relevant degree because of a blog post making the point that we're mostly identical in obvious ways? (ASPD and autism would seem -- I'm not sure what you mean by relevant -- to discount conflicting terminal values not existing to a relevant degree, and then there's enculturation/learning/environment which all affect the brain. Human universals are really cultural universals. To see human universals look towards feral children. This makes your learned values claim suspicious.)
So, assuming you're right, I think your conclusion then is that it's more productive to work towards uncovering what would be reflective extrapolated values than it is to bargain, but that's non-obvious given how political even LWers are. But OTOH I don't think we have anything to explicitly bargain with.
everyone wants others to be hated
Speak for yourself.
At a high level of generality, people with different values will use the same words to articulate them. But at that level, the assertions are merely applause lights. The ambiguity serves to hide real disputes.
When the discussion down to object-level disputes, the different meanings very quickly devolve into different choices. For example, the discussion about "creepy" behavior was in part a discussion about what behaviors were, and were not, oppressive. And who gets to make that judgment.
Another way of looking at the "Don't be creepy" discussion is that some folks were saying "XYZ behavior is oppressive," while other groups were saying "No it isn't."
As you say, everyone thinks oppressive behavior should stop. My point was that one's definition of oppressive relies on one's terminal values.
In other words, you said:
I don't think we are anywhere near the point where fundamental value differences between [different groups] are relevant.
I think that assertion is empirically false.
I think that assertion is empirically false.
You have some evidence that I don't or we are using "fundamental" differently.
My "fundamental" may be a bad concept, but what's your reason for thinking humans have irreconcilable value differences more significant than stuff like details of aesthetic taste?
In brief, the universality of the politics-is-the-mindkiller phenomena. If some ideologies or political topics were more likely to reach agreement than others, that would be evidence that some terminal value differences are not "fundamental".
And there aren't any areas of universal agreement. It's pretty easy for someone to find a viable society that supported just about any terminal value one could suggest.
In brief, the universality of the politics-is-the-mindkiller phenomena.
People seem universally drawn to status debates and politics. This is evidence against uniform fundamental values?
If some ideologies or political topics were more likely to reach agreement than others, that would be evidence that some terminal value differences are not "fundamental".
How does this work? Can you expand?
On more thot, I retract the "people don't 'fundamentally' disagree" thing. Seems awfully strong now, especially when a good chunk of who we are is memetic and not just genetic. Also, 'fundamentally' is a confused concept among humans.
Still, I hold that people leap to "fundamental value differences" as an explanation far too easily. Seems too convenient (It's ok, a peaceful solution will never work and we have to kill them because Fundamental Value Differences) and comes to mind too easily for self-serving and confused reasons (reifying an unproductive argument as a Fundamental Value DIfference is a nice comfortable solution that has no reason to be correct)
A little breath of sanity in political discourse. How refreshing.
Value is subjective. What I value isn't exactly what you value. That's the start of a meaningful political discussion, instead of all this piffle about what "we" want.
And your point about potential benefits of self and thereby other deception in politics makes sense to me, though it's unclear that delusion as a policy will be, on balance, more effective. But that's an empirical and contextual matter, not one that's solved by sitting around thinking about it.
But if you want to talk politics of these differing subjective values, I'd recommend starting with the premise that delusion is at least unhelpful in modeling and understanding politics, whatever use it might have as a tool in politics itself.
Did you just succeed in using gender conflicts as the non political analogous example which allows rational discourse regarding a highly inflamed, trench war topic that would degenerate into something worse than a (subtle and cold version of a) flame war if discussed directly? (different estimates of ones type of reflective equilibrium results in different preferred extrapolation dynamic/initial group/etc (which of course results in cases where it can be instrumentally rational for a non perfect liar to believe in false things))
If you did this on purpose, you are my new personal hero!
The relevance to CEV/FAI did cross my mind when I was writing this post, but no, the reason I wrote it was that people were saying that the gender-related discussions on LW are not very productive, but seemed puzzled as to why.
What counts as "productive"? If the problem is under-acknowledged value differences, then perhaps part of the solution is to air differences of values. That seems like the sort of thing that would look like a lot of disagreement and assertion.
For example, when discussing gender-related problems, it seems inevitable that some proposed solutions will generally be better for men, and other solutions will generally be better for women. If people are selfish, then they will each prefer the solution that's individually best for them, even if they can agree on all of the facts. (It's unclear whether people should be selfish, but it seems best to assume that most are, for practical purposes.)
But isn't it possible that in any given bargaining situation there may also be a win-win solution that makes the pie bigger and leaves everybody better off than the status quo? Discussion, debate, and further exchange of information could at least theoretically lead to a previously unrecognized win-win situation being found.
"Leaves everyone better off" (i.e., making a Pareto improvement) is a tall order if there are more than a few people involved. Just "making the pie bigger" is much more plausible, but in general people will disagree about what counts as making the pie bigger, since we don't have an agreed-upon way of doing interpersonal comparison of utility.
I'm not saying that discussion/debate can't serve the purpose of joint optimization, but it often doesn't, and we can mostly see why.
It's worth noting that the values folks are programmed to "bargain" with may be unendorsed leftovers from evolution. So maybe it's possible to make progress in this sort of discussion if both parties are trying to refactor their values?
It's worth noting that the values folks are programmed to "bargain" with may be unendorsed leftovers from evolution.
Good point. A good reason to be less biased even if that's not "instrumentally rational" is that your biases may only be rational for a set of values that aren't actually yours in an important sense.
I think this goes for "men versus women" less than it does for any other coalitional conflict. Men and women often have their interests aligned: for instance friendships between the sexes are common and family relationships between the sexes are universal. The suggestion that women should form a coalition and seek concessions from men is sometimes called "feminism," but according to this poll this idea is unpopular in the US. The suggestion that men should form a coalition and seek concessions from women is almost never made explicitly except by some crackpots. (I do understand that many people believe there is a very powerful but less explicit coalition, perhaps called "patriarchy"). For flemish versus french belgians, it's another story.
Good points, but I think "try to empathize with those who are of a different race/nationality than you" is a stronger meme than "try to empathize with those who are of a different gender than you". At least where I live (US).
I think the problem is that where you can openly describe the conflicts, you can also openly propose game-theoretical solutions. You can say: "We should live in peace with people of Sylvania... but if they attack us, we will not hesitate to fight against them too." And then you can explain why you think this is a good strategy, instead of e.g. us attacking first.
In the gender questions today, we are not culturally allowed to describe the nature of the conflict, which is: different reproduction mechanisms leading to different reproduction strategies leading to conflicts of interest.
So instead of stating our true interests, and negotiating about them, we speak about something else. For example: "It is good if a man must pay child support for a child that is not biologically his, because such policy is good for children." (Instead of admitting openly that such policy allows a woman to increase her utility function, because she does not have to compromise in her choice of partner between his attractiveness and responsibility, and can maximize for attractiveness instead.) On the other hand, forbidding women access to higher education could also be framed as good for children... except that this kind of re-framing was already thoroughly exposed by feminists.
It is difficult to propose a policy of "I will cooperate in Prisonners' Dillemma if and only if you will cooperate", if in the first place you are not allowed to admit that the conflict exists, and if speaking about the payoff matrices is such strong taboo that many people even don't know them.
So instead we randomly optimize for children in some places, and against children in other places, pretending that this is all done around the adult-child axis, and completely unrelated to man-woman axis. (It's adults who have a right for complete freedom of their bodies and everything that happens to be inside; and it's children who have a right for income proportional to their biological father's income. See, this is almost completely gender neutral! But if you try to suggest that instead the children should have a right to live, and adults should have a right for financial freedom, it is culturally allowed to expose how sexist your suggestions are.)
"Not culturally allowed"? What you are describing is the mainstream cultural view of gender and sex roles!
In gist: There are two kinds of people, men and women; God or Nature has given them different sexual likes; notably, men pursue and women are pursued; businessmen want to chase after hot young things in short skirts while secretaries want to cheat with their bosses. Men are supposed to be go-getters and bring home the bacon; women are supposed to be pretty and nice and take care of the kids. (Unsatisfied men become aggressive and abusive; unsatisfied women become passive-aggressive and bitchy. A bad man is a murderer or rapist; a bad woman is a slut or a scold.) Happy and stable relationships come from acknowledging that these fundamental differences are normal, natural, fair, morally good, and unchangeable — and negotiating within them.
This is totally culturally allowed. It's on daytime TV, popular self-help books, and the plots of romantic comedies. You're swimming in it.
(It's also a bunch of what all that feminism and queer theory and such are responding to. They call it things like "patriarchy" and "gender essentialism" and "heteronormativity".)
There are complex rules about what are you allowed to say where. Saying something as a joke a in romantic comedy may increase your revenues; saying the same thing in academy can get you fired. I should have added the proper disclaimers.
The same message can be at the same time in the same country both popular and forbidden, both mainstream and ostracized, both "of course everybody knows this" and "you must be an evil mutant if you even think about this".
In their private lives many people find a win-win solution. Also, many don't. Even without the explicit game theory people gradually gain some insight that (1) with some people pressing the "cooperate" button brings you horrible outcomes, but (2) pressing the "defect" button almost always brings bad long-term outcome; therefore the key to long-term happiness is pressing the "cooperate" button with a person who is able and willing to respond properly.
But even then there is the nagging feeling that your outcome could have been better if you just were allowed to press "defect" while the other person would for some reason press their "cooperate" button. And there are often friendly people telling you that this is exactly what you deserve, and that you should be true to thyself and never compromise for other than the greatest value in your payoff matrix.
Of course the problem is that the buttons are not labelled "cooperate" and "defect" on both sides; that would make the whole situation much easier. Instead, my "cooperate" button comes with a label "Sacrifice", and the "defect" button comes with a label "Freedom". I don't clearly see what is written on your buttons, but considering the effect they have on me, I simply call your "cooperate" button the good button, and your "defect" button the bad button. Therefore it seems obvious to me that you have a moral obligation to push the good button, while I have a moral right to push the freedom button. And I don't understand why do you always have to discuss this obviously superior solution.
This is what being in a Prisoners' Dilemma with unacknowledged different values feels like from inside.
My point was that some sort of gender essentialism ("men and women necessarily have different values") and heteronormativity ("the ideal outcome for every man and every woman is a Cooperate/Cooperate pairing with a member of the opposite sex") are so mainstream as to be almost entirely uncontroversial until rather recently.
In many contexts, anyone who complains about them, or considers that the world could be otherwise, is dismissed as a loser incapable of dealing with reality. (Whether male or female; one word for a female loser is "ugly", while one word for a male loser is "immature" and another is "whipped".)
However, to stick with your Prisoner's Dilemma theme: Part of the point of the feminist idea of "patriarchy" is that the deal that women get out of the traditional arrangement is not a nice strategy like tit-for-tat. The strategy that patriarchy teaches men to follow in their relations with women is more like an extortionate strategy, where the expected penalties for non-cooperation are much greater for women. ("Men are afraid women will laugh at them; women are afraid men will kill them.")
One thing I wonder about genetic essentialism on sexual behavior is how many generations it would take for moral changes at the social level to be driven into the genome. Here's a nasty, nasty thought-experiment: Suppose there is (or can be) genetic variation in the tendency of men to rape women. In a society where rape victims are made to bear their assailants' children, "rape genes" could be quite favored by evolution. In a society where rape victims reliably get abortions, and rapists are castrated or executed, "rape genes" would be extinguished — and a mutation that made rape less likely would be favored.
How's that? Can I get some of those taboo points? :)
However, to stick with your Prisoner's Dilemma theme: Part of the point of the feminist idea of "patriarchy" is that the deal that women get out of the traditional arrangement is not a nice strategy like tit-for-tat. The strategy that patriarchy teaches men to follow in their relations with women is more like an extortionate strategy, where the expected penalties for non-cooperation are much greater for women. ("Men are afraid women will laugh at them; women are afraid men will kill them.")
Men are also afraid of being killed by men. US statistics here for which sex has more to fear from men. "The battle of the sexes" is not a two player game; I think analogies to the prisoner's dilemma are misleading.
One thing I wonder about genetic essentialism on sexual behavior is how many generations it would take for moral changes at the social level to be driven into the genome. Here's a nasty, nasty thought-experiment: Suppose there is (or can be) genetic variation in the tendency of men to rape women. In a society where rape victims are made to bear their assailants' children, "rape genes" could be quite favored by evolution. In a society where rape victims reliably get abortions, and rapists are castrated or executed, "rape genes" would be extinguished — and a mutation that made rape less likely would be favored.
If you replace "rape" by more general antisocial behavior, there is some discussion about whether this has been happening in Pinker's "Angels."
A bad man is a murderer or rapist; a bad woman is a slut or a scold.
For the Victorians, the go-to feminine stereotype to match "murderer or rapist" was "poisoner."
You're making assertions which I'm not sure they're nearly as obvious and universal nowadays as you think they are. Perhaps they're still true more often than not -- but I'd assign roughly 50/50 which direction the stereotypes you gave nowadays go.
E.g. on my part I don't watch TV often, but the last time I saw in comedies businesspeople chase after their "hot young" secretaries, was when in "Friends" Rachel was chasing after her male secretary, and Chandler's female boss was chasing after him. Just a datapoint, perhaps you have more recent and more frequent "daytime TV" and romantic comedies examples?
On the other hand, forbidding women access to higher education could also be framed as good for children...
Er... How so? I can't see any reason why I'd rather my mother was less educated than she actually is.
It should only take a second or two to think of possible reasons.
Here's one: Education causes birth rates to plunge. So if existence is a good, then the more you educate women, the more you harm children.
Yeah, I hadn't thought about that -- though IIRC educated men also have fewer children in average. (And personally I lean more to average utilitarianism than total utilitarianism at the moment.)
Men don't get hit as hard, IIRC. And it's not like men have ever been bottlenecks to reproduction - an educated woman can just go use a sperm bank (significantly better than an old husband in terms of birth defects), engage one-night stands, marry downwards, etc.
How should people bargain in an online and mostly anonymous forum? A sort of reality-disconnected sense of status is all there is to bargain with, and status manipulation is more unconscious and implicit than what might be had by bargaining. I think all we can hope for is to fight and destroy those with conflicting values in communal status warfare, and then bring back the destroyed as amusing contrarians.
And foregoing the effort because it's unproductive or lowers the "sanity level" is just so very boring.
For example, when discussing gender-related problems, [edit] one solution may be generally better for men, while another solution may be generally better for women
Love is war.
All is fair in love and war.
Individually optimal behavior by each male doing what is best for himself, and each female doing what is best for herself, is unlikely to be optimal as for males and females as a whole, or even particular male/female couples.
Such prisoners dilemma problems are normally solved by coercion - chastity imposed on females, shotgun marriages and continued material support imposed on males.
However, I am fairly sure that were I to discuss the usual solutions in detail, the response would be strikingly lacking in rationality.
Love is war.
All is fair in love and war.
The second of those quotes is superior (ie. less denotatively insane) but I note that if you have already assumed the first quote half of the second is redundant.
Value differences are under-acknowledged because our society makes no real attempt to elicit individual values. There is very little for any individual to gain by speaking about his/her true values or beliefs. The result of such a confession of true personal beliefs is almost invariably that the person in question will be subjected to scathing socio-political attack. Consider the examples of John Mackey, Larry Summers, Dan Cathy, and Todd Aikin, to name a few that come easily to mind.
Just to clarify... do you mean to suggest that the four people you list are typical of the degree to which expressing true personal beliefs leads to sociopolitical attack? As in, for example, if I were to foolishly post to my personal blog about my true personal beliefs I should expect that level of attack in response?
You should certainly be careful and concerned about what you post on your personal blog if your opinions are out of synch with mainstream. Obviously prominent individuals attract a greater total level of criticism, but expressing an unorthodox opinion could certainly cost you your job, for example.
Ah, OK. Sure, if we're restricting ourselves to people whose opinions are sufficiently out of synch with the mainstream, I expect something like this is true.
Thanks for clarifying.
If one considers all of their beliefs it's likely they have at least one which is extremely out of synch with the mainstream. The non-caricatured version of Larry Summers opinion was not literally insane. It is a widely-held belief among some populations. If the average Less Wrong user faced the same level of scrutiny and had to give a complete index of their beliefs, it seems almost certain they would face similar consequences.
The non-caricatured version of Larry Summers opinion was not literally insane.
Let's not forget that it even wasn't his stated opinion. It was one of the three hypotheses he suggested as worth testing experimentally:
- The high-powered job hypothesis
- Different availability of aptitude at the high end
- Different socialization and patterns of discrimination in a search
So more precisely, you can be fired even for a non-mainstream opinion you don't express, as long as people try hard enough to assign it to you connotationally. When the social norm is to disbelieve X, then saying "we should test X experimentally" suggests that you are not disbelieving enough; otherwise you would consider experimentally testing X a waste of time.
Also, when enough people consider powerful people who believe X and act accordingly a personal threat to them or people they care about, being perceived as a powerful person expressing a willingness to consider X causes lots of people to treat me as a personal threat.
Sure; I would agree that if we (somehow) faced the same level of scrutiny as Larry Summers, we'd likely face similar consequences for our own quirky beliefs. But I also agree with Daniel that prominent individuals attract a greater total level of criticism, which makes the premise implausible.
What I was trying to clarify was whether we were talking about typical scenarios. From Daniel's response I conclude they weren't. To clarify I'll ask you the same question: do you mean to suggest that Larry Summers is typical of the degree to which expressing true personal beliefs leads to sociopolitical attack? As in, for example, if I were to foolishly post to my personal blog about my true personal beliefs I should expect that level of attack in response?
As in, for example, if I were to foolishly post to my personal blog about my true personal beliefs I should expect that level of attack in response?
Of course not. You would receive a lesser response. Those are more like worst case scenarios. The "typical" response of me posting my most non-mainstream views on a personally-identified public blog is that future employers google my name after interviewing me, see that my beliefs differ dramatically from their own, and a certain percentage choose not to pursue any further discussions with me (whether consciously or subconsciously due to lowered perceived value), decreasing my prospects in the labor markets. This matters enough to me that I make specific efforts not to have non-mainstream beliefs associated with my personal information.
Grad students who make up a disproportionate quantity of Less Wrong perhaps can afford to care a lot less about this than others. Undergrads at Less Wrong may lack appropriate perspective to understand the influence this could have on their future. On the other end of the spectrum, no lawyer in his right mind would ever consider posting about such things. They are a worst case scenario because reputation is their only asset and they are essentially "hired" by every client, dozens of times per year.
Thanks for the clarification.
For my own part, I doubt that my most nonmainstream views would, were future employers to encounter my articulation of them in any sort of representative context, affect their decision to hire me more than, say, my taste in poetry does. (If it matters, I'm a software professional in my 40s.)
Grad students who make up a disproportionate quantity of Less Wrong perhaps can afford to care a lot less about this than others.
Yep. Once a professor of mine told me “I know your English is good because I saw the stuff you used to write on Facebook when you were in Ireland” -- and much of that was stuff that most other people would find quite embarrassing and wouldn't post under their real name where their superiors could see it.
I've been reading a lot of the recent LW discussions on politics and gender, and noticed that people rarely bring up or explicitly acknowledge that different people affected by some political or gender issue have different values/preferences, and therefore solving the problem involves a strong element of bargaining and is not just a matter of straightforward optimization. Instead, we tend to talk as if there is some way to solve the problem that's best for everyone, and that rational discussion will bring us closer to finding that one best solution.
For example, when discussing gender-related problems, one solution may be generally better for men, while another solution may be generally better for women. If people are selfish, then they will each prefer the solution that's individually best for them, even if they can agree on all of the facts. (It's unclear whether people should be selfish, but it seems best to assume that most are, for practical purposes.)
Unfortunately, in bargaining situations, epistemic rationality is not necessarily instrumentally rational. In general, convincing others of a falsehood can be useful for moving the negotiated outcome closer to one's own preferences and away from others', and this may be done more easily if one honestly believes the falsehood. (One of these falsehoods may be, for example, "My preferred solution is best for everyone.") Given these (subconsciously or evolutionarily processed) incentives, it seems reasonable to think that the more solving a problem resembles bargaining, the more likely we are to be epistemicaly irrationality when thinking and talking about it.
If we do not acknowledge and keep in mind that we are in a bargaining situation, then we are less likely to detect such failures of epistemic rationality, especially in ourselves. We're also less likely to see that there's an element of Prisoner's Dilemma in participating in such debates: your effort to convince people to adopt your preferred solution is costly (in time and in your and LW's overall sanity level) but may achieve little because someone else is making an opposite argument. Both of you may be better off if neither engaged in the debate.