Would a perfectly rational B be offended at all by an incorrect guess?
I eventually come out with a contingent "yes" to this question, but it took me a while to get there, and I don't entirely trust my reasoning.
As stated, I wasn't sure how to go about answering that question.
But when A guesses about B, this reveals facts about A's priors with respect to B. So this question seemed isomorphic to "Would B be offended by A believing certain things about B?" which seemed a little more accessible.
But I wasn't exactly sure what "offended" means, at this level of description. The best unpacking I could come up with was that I'm offended by an expressed belief when I subconsciously or instinctively choose to signal my strong rejection of that belief.
If that's true, then I can rephrase the question as "Would a perfectly rational B subconsciously or instinctively choose to signal strong rejection of certain beliefs about B?"
If B has saliently limited conscious processing ability (limited either by speed or capacity) then my answer is a contingent "yes."
For example, a perfectly rational B might reason as follows: "Consider the proposition P1: 'B is willing to cheat'. Within a community that lends weight to my signaling, there is value to my signaling a strong rejection of P1. Expressing offense at P1 signals that rejection. Expressing offense successfully depends on very rapid response; if I am seen as taking time to think about it first, my offense won't signal as effectively. So I do better to not think about it first, but instead instinctively express offense without thinking. In other words, I do better to be offended by the suggestion of P1. OK, let me go implement that."
In this example, B's conscious processing speed forms the salient limitation, but what's important here is the general condition that an unconscious result has value relative to a conscious one.
The specific value provided in this example is less important; there are lots of different equivalent examples.
During discussion in my previous post, when we touched the subject of human statistical majorities, I had a side-thought. If taking the Less Wrong audience as an example, the statistics say that any given participant is strongly likely to be white, male, atheist, and well, just going by general human statistics, probably heterosexual.
But in my actual interaction, I've taken as a rule not to make any assumptions about the other person. Does it mean, I thought, that I reset my prior probabilities, and consciously choose to discard information? Not relying on implicit assumptions seems the socially right thing to do, I thought; but is it rational?
When I discussed it on IRC, this quote by sh struck me as insightful:
I came up with the following payoff matrix:
In this case, the second option is strictly preferable. In other words, I don't discard the information, but the repercussions to our social interaction in case of an incorrect guess outweigh the benefit from guessing correctly. And it also matters whether either Alice or Bob is an Asker or a Guesser.
One consequence I can think of is that with a sufficiently low p, or if Bob wouldn't be particularly offended by Alice's incorrect guess, taking the guess would be preferable. Now I wonder if we do that a lot in daily life with issues we don't consider controversial ("hmm, are you from my country/state too?"), and if all the "you're overreacting/too sensitive" complaints come from Alice incorrectly assessing a too low-by-absolute-value negative payoff in (0, 1).