A related thought about the act of revealing your beliefs: what expected utility would you assign to shouting "The Emperor has no clothes!" when the Emperor, indeed, has no clothes?
If you're selfish, you would probably refrain from it under fear of punishment.
But even if you care about the overall utility of the society rather than your personal utility, revealing the truth about the Emperor's clothes could cause rebellions and anarchy, or collapse of whole markets and fields of discourse centered about His Majesty's supposed sophisticated attire.
Does this mean that we can't factor politics completely out of rationality discussions, even when dealing with facts that are objectively and unambiguously true? In "Three Worlds Collide", for example, there is a backstory element where scientists chose to suppress a scientific discovery in the fear of disastrous social effects if it was made public.
A related thought about the act of revealing your beliefs: what expected utility would you assign to shouting "The Emperor has no clothes!" when the Emperor, indeed, has no clothes?
Maybe I'm big on virtue ethics and consider always striving to speak truth virtuous. What if I think me or society "living a lie" is something bad? People have been willing to die painfully and ferment social unrest because of what they consider truth since ... forever, so I'm not even sure its that exotic a mindset.
During discussion in my previous post, when we touched the subject of human statistical majorities, I had a side-thought. If taking the Less Wrong audience as an example, the statistics say that any given participant is strongly likely to be white, male, atheist, and well, just going by general human statistics, probably heterosexual.
But in my actual interaction, I've taken as a rule not to make any assumptions about the other person. Does it mean, I thought, that I reset my prior probabilities, and consciously choose to discard information? Not relying on implicit assumptions seems the socially right thing to do, I thought; but is it rational?
When I discussed it on IRC, this quote by sh struck me as insightful:
I came up with the following payoff matrix:
In this case, the second option is strictly preferable. In other words, I don't discard the information, but the repercussions to our social interaction in case of an incorrect guess outweigh the benefit from guessing correctly. And it also matters whether either Alice or Bob is an Asker or a Guesser.
One consequence I can think of is that with a sufficiently low p, or if Bob wouldn't be particularly offended by Alice's incorrect guess, taking the guess would be preferable. Now I wonder if we do that a lot in daily life with issues we don't consider controversial ("hmm, are you from my country/state too?"), and if all the "you're overreacting/too sensitive" complaints come from Alice incorrectly assessing a too low-by-absolute-value negative payoff in (0, 1).