This looks like a special case of a failure of intentionality. If a child knows where the marble is, they've managed first-order intentionality, but if they don't realize that Sally doesn't know where the marble is, they've failed at second order.
The orders go higher, though, and it's not obvious how much higher humans can naturally go. If
Bob thinks about "What does Alice think about Bob?" and on rare occasions "What does Alice think Bob thinks about Alice?" but will not organically reason "What does Alice think Bob thinks about Alice's model of Bob?"
then Bob can handle second and third but can't easily handle fourth order intentionality.
It may be a useful writing skill to be comfortable with intentionality at one level higher than your audience.
but will not organically reason "What does Alice think Bob thinks about Alice's model of Bob?"
That's actually pretty easy: Alice doesn't :-)
Obligatory reference: Battle of Wits.
Cross-posted from my blog
I'd like to coin a term. The Sally-Anne fallacy is the mistake of assuming that somone believes something, simply because that thing is true.1
The name comes from the Sally-Anne test, used in developmental psychology to detect theory of mind. Someone who lacks theory of mind will fail the Sally-Anne test, thinking that Sally knows where the marble is. The Sally-Anne fallacy is also a failure of theory of mind.
In internet arguments, this will often come up as part of a chain of reasoning, such as: you think X; X implies Y; therefore you think Y. Or: you support X; X leads to Y; therefore you support Y.2
So for example, we have this complaint about the words "African dialect" used in Age of Ultron. The argument goes: a dialect is a variation on a language, therefore Marvel thinks "African" is a language.
You think "African" has dialects; "has dialects" implies "is a language"; therefore you think "African" is a language.
Or maybe Marvel just doesn't know what a "dialect" is.
This is also a mistake I was pointing at in Fascists and Rakes. You think it's okay to eat tic-tacs; tic-tacs are sentient; therefore you think it's okay to eat sentient things. Versus: you think I should be forbidden from eating tic-tacs; tic-tacs are nonsentient; therefore you think I should be forbidden from eating nonsentient things. No, in both cases the defendant is just wrong about whether tic-tacs are sentient.
Many political conflicts include arguments that look like this. You fight our cause; our cause is the cause of [good thing]; therefore you oppose [good thing]. Sometimes people disagree about what's good, but sometimes they just disagree about how to get there, and think that a cause is harmful to its stated goals. Thus, liberals and libertarians symmetrically accuse each other of not caring about the poor.3
If you want to convince someone to change their mind, it's important to know what they're wrong about. The Sally-Anne fallacy causes us to mistarget our counterarguments, and to mistake potential allies for inevitable enemies.
From the outside, this looks like "simply because you believe that thing".
Another possible misunderstanding here, is if you agree that X leads to Y and Y is bad, but still think X is worth it.
Of course, sometimes people will pretend not to believe the obvious truth so that they can further their dastardly ends. But sometimes they're just wrong. And sometimes they'll be right, and the obvious truth will be untrue.