The rest of your brain and psychology evolved under the assumption that you would have a functioning oxytocin receptor, so I think there's an a priori case that it would be beneficial for you if it worked properly(yes, evolution's goals are not identical to your own, still though.....)

One of the reasons it's hard to take the possibility of blatant lies into account is that it would just be so very inconvenient, and also boring. If someone's statements are connected to reality, that gives you something to do: * You can analyze them, critique them, use them to infer the person's underlying models and critique those, identify points of agreement and controversy, identify flaws in their thinking, make predictions and projections about future actions based on them, et cetera. All those activities we love a lot, they're fun and feel useful. * It also gives you the opportunity to engage, to socialize with the person by arguing with them, or with others by discussing their words (e. g., if it's a high-profile public person). You can show off your attention to detail and analytical prowess, build reputation and status. On the other hand, if you assume that someone is lying (in a competent way where you can't easily identify what are the lies), that gives you... pretty much nothing to do. You're treating their words as containing ~zero information, so you (1) can't use them as an excuse to run some fun analyses/projections, (2) can't use them as an opportunity to socialize and show off. All you can do is stand in the corner repeating the same thing, "this person is probably lying, do not believe them", over and over again, while others get to have fun. It's terrible. Concrete example: Sam Altman. The guy would go on an interview or post some take on Twitter, and people would start breaking what he said down, pointing out what he gets right/wrong, discussing his character and vision and the X-risk implications, etc. And I would listen to the interview and read the analyses, and my main takeaway would be, "man, 90% of this is probably just lies completely decoupled from the underlying reality". And then I have nothing to do. Importantly: this potentially creates a community bias in favor of naivete (at least towards public figures). People who believe that Alice is a liar mostly ignore Alice,[1] so all analyses of Alice's words mostly come from people who put some stock in them. This creates a selection effect where the vast majority of Alice-related discussion is by people who don't dismiss her words out of hand, which makes it seem as though the community thinks Alice is trustworthy. That (1) skews your model of the community, and (2) may be taken as evidence that Alice is trustworthy by non-informed community members, who would then start trusting her and discussing her words, creating a feedback loop. Edit: Hm, come to think of it, this point generalizes. Suppose we have two models of some phenomenon, A and B. Under A, the world frequently generates prompts for intelligent discussion, whereas discussion-prompts for B are much sparser. This would create an apparent community bias in favor of A: A-proponents would be generating most of the discussions, raising A's visibility, and also get more opportunities for raising their own visibility and reputation. Note that this is completely decoupled from whether the aggregate evidence is in favor of A or B; the volume of information generated about A artificially raises its profile. Example: disagreements regarding whether studying LLMs bears on the question of ASI alignment or not. People who pay attention to the results in that sphere get to have tons of intelligent discussions about an ever-growing pile of experiments and techniques. People who think LLM alignment is irrelevant mostly stay quiet, or retread the same few points they have for the hundredth time.  1. ^ What else they're supposed to do? Their only message is "Alice is a liar", and butting into conversations just to repeat this well-discussed, conversation-killing point wouldn't feel particularly productive and would quickly start annoying people.

> did not listen to me saying I did not think Y. But it really seems like you do have a significant disagreement with Dai about the extent to which deference to Yudkowsky was justified. I understand and acknowledge that you think deference has large costs, as you've previously written about. I also understand and acknowledge that you think defering to Yudkowsky on existential risk strategy in particular was costly, as you explicitly wrote in the post ("one of those founder effects was to overinvest in technical research and underinvest in 'social victory' [...] Whose fault was that?"). At the same time, however, in your discussion in the post of how people could have done better in that particular case, you emphasize being transparent about deference in order to reduce its distortionary effects ("We don't have to stop deferring, to avoid this correlated failure"), in contrast to how Dai argues in the comment section that not-deferring was a live option ("These seemed like obvious mistakes even at the time"). You even seem to ridicule Dai for this ("And then you're like 'Ha. Why not just not defer?'"). This seems like a real and substantive disagreement, not a hallucination on Dai's part. It can't simultaneously be the case that Dai is wrong to implicitly ask "Why not just not defer?", and also wrong to suggest that you disagree with him about when it's reasonable to defer.