That may well be true, but I should clarify that neither of my hypotheticals require or suggest that bad faith communication was more common in the past. They do suggest that assumptions of bad faith may have been significantly more common than actual bad faith, and that this hypersensitivity may have been adaptive in the ancestral environment but be maladaptive now.

It would be surprising, if bad intent were so rare in the relevant sense, that people would be so quick to jump to the conclusion that it is present. Why would that be adaptive?

You may not be wrong but I don't think it would necessarily be surprising. We adapted under social conditions that are radically different than exist today. It may no longer be adaptive.

Hypothesis: In small tribes and family groups assumptions of bad faith may have served to help negotiate away from unreasonable positions while strong familial ties and respected third parties mostly mitigated the harms. Conflicts between tribes without familial connections may have tended to escalate however (although there are ways to mitigate against this too).

Hypothesis: Perhaps assumptions of good and bad faith were reasonably accurate in small tribal and familial groups but in intertribal disagreements there was a tendency to assume bad faith because the cost of assuming good faith and being wrong was so much higher than assuming bad faith and being wrong.

I really liked this post. I thought it was well written and thought provoking.

I do want to push back a bit on one thing though. You write:

What makes for a crony belief is how we're rewarded for it. And the problem with beliefs about climate change is that we have no way to act on them — by which I mean there are no actions we can take whose payoffs (for us as individuals) depend on whether our beliefs are true or false.

It is true that most of us probably won't take actions whose payoffs depend on beliefs about global warming, but it is not true that there are no such actions. One could simply make bets about the future global average temperature.

So the problem is not that there are no actions we can take whose payoffs depend on whether our beliefs are true or false. Rather beliefs about global warming are likely to be cronies because the subject has become highly political. And as you correctly point out, in politics social rewards completely dominate pragmatic rewards.

To illustrate, it is even harder to find actions we can take whose payoffs depend on the accuracy of the belief that the Great Red Spot is a persistent anticyclonic storm on the planet Jupiter. Does this mean that a belief in the Great Red Spot is even more likely to be cronyistic than a belief regarding global warming?

Thanks, I did end up figuring out my error.

Maybe I'm confused, in the 'muddy children puzzle' it seems it would be common knowledge from the start that at least 98 children have muddy foreheads. Each child sees 99 muddy foreheads. Each child could reason that every other child must see at least 98 muddy foreheads. 100 minus their own forehead which they cannot see minus the other child's forehead which the other child cannot see equals 98.

What am I missing?

Desire is a contract you make with yourself to be unhappy until you get what you want.

– Naval Ravikant

You can see more results here: Image Annotation Viewer

Judging generously, but based on only about two dozen or so image captions, I estimate it gives a passably accurate caption about one third of the time. This may be impressive given the simplicity of the model, but it doesn't seem unreasonably effective to me, and I don't immediately see the relevance to strong AI.

Let's say you precommit to never paying off blackmailers. The advantage of this is that you are no longer an attractive target for blackmailers since they will never get paid off. However if someone blackmails you anyway, your precommitment now puts you at a disadvantage, so now (NDT)you would act as if you had a precommitment to comply with the blackmailers all along since at this point that would be an advantageous precommitment to have made.

Audio here, Mitchell and Webb - The Boy Who Cried Wolf.

It's a funny joke but beside the point. Knowing that he is in a balloon about 30 feet above a field is actually very useful. It's just useless to tell him what he clearly already knows.