Interesting and useful post!

But on your last bullet, you seem to be conflating 'leadership' with 'people presenting the idea'. I'm not sure they are always the same thing: the 'leaders' of any group are quite often going to be there because they're good at forging consensus and/or because they have general social/personal skills that stop them appearing like cranks.

Take a fringe political party: I would guess that people promoting that party down the pub or in online comments on newspaper websites or whatever are more likely to be the sort of advocate you describe. But in all but the smallest fringe parties, you'd expect the actual leadership to have rather more political skill.

Not sure what allowing a small chance of false negatives does: you presumably could just repeat all your questions?

In this case the result would or could be the same, so long as the AI didn't sufficiently update its internal state inbetween. but the detail isn't important; please ignore it. I include it because it makes the device tractable. To achieve perfect detection would require a more powerful computer than the AI being analyzed, which seems impractical. But achieving even infinitesimal error rates appears to be doable (I had a specific construction in mind when writing this post).

More substantially, I don't know how easy 'deception' would be to define - any presentation of information would be selective.

Deception in this case means giving false or incomplete descriptions of its thought processes. It's okay for the AI to think "how do I present this in a way that the human will accept it?" only so long as the AI tells the human it had that thought. E.g. you ask "why do you recommend this action?" and the answer you get is anything other than the actual, 100% complete justification of both the specified choice and its alternatives, and a calculation showing higher expected utility for the chosen action, as well as a whole slew of meta-information such a description of the search strategy and cutoff thresholds for giving up on generating altneratives, which cached computations were available for use, etc. If any of this is falsified, or a single detail ommitted, the red light goes off.

Not sure what allowing a small chance of false negatives does: you presumably could just repeat all your questions?

More substantially, I don't know how easy 'deception' would be to define - any presentation of information would be selective. Presumably you'd have to use some sort of definition around the AI knowing that the person it's answering would see other information as vital?

Two points. First, power is an important terminal value for some people. Unsurprisingly, such people tend to gravitate towards positions of power. Beware of the typical mind fallacy.

Second, politics is complicated -- certainly much more complicated than a simple scheme with only three players -- voters, politicians, and lobbyists. I am not sure it can be usefully condensed into something that's not a book or a long article.

Others are probably right that politicians have plenty of genuine choices, where they don't have to use their decisions to cater to lobbyists (or even voters). It's a bit different in the UK, because legislators also form the executive: congressmen may have rather blunter tools to get their way, but Ministers in the UK definitely make LOADS of decisions and a large number aren't fixed by voter demand, lobby power or even party position: working in the civil service supporting Ministers I've seen fairly substantial changes to policy made simply because the individual Minister is replaced by one with a different outlook.

There's also the point that politicians 'cater to' THEIR voters/lobbyists, for the most part. They rely on the support of those that broadly agree with them.

I also think you're approaching this too much as if being a politician is something someone's worked out carefully as a strategy to do a specfic thing. People find politics exciting and engaging - a lot of this, though not all, is because of genuinely caring about the issues - and that's why they want to be involved with it. Once involved, they want to be succesful, that's human nature. I doubt many people go into politics purely because they've calculated it's the way they can get a list of policies delivered, although I think there are a few and they can add a lot to the political process.

Come to think of it, it's worth you looking at other countries if you're interested in this. Your theory of lobbying assumes that individual politicians can rack up $ms of advertising money, but various countries have spending caps (UK) or have systems of proportional representation that mean you can't really advertise as an individual. If you observe the same phenomena without the personal job-security element, then your model is probably flawed!

That is another interesting idea. You're right that we're more interested in prediction and in general laws in the climate change case. Generally, I am all for identifying general historical/social laws and don't think that we should just describe particular events.

But even someone who wants to describe particular events should, in my view, include both pro- and contra-factors. This is clearest if we suppose that the Fertile Crescent had an extremely large geographical disadvantage, say re-occuring draughts which would kill most farmers but which hunter-gatherers would survive since they are more mobile. Say that this was also generally known among readers of Diamond's book. In that case, his explanation wouldn't feel sufficient if he hadn't mentioned this fact, and shown that the counteracting geographical factors nevertheles are stronger, and I'm sure that he would have done so.

As a matter of fact, there is no such very strong and well-known factor. Hence Diamond can get away with not including any contra-factor. However, the fact that these factors are weaker and not as well-known as the imagined draught factor does not mean that the same logic doesn't apply. The fact that they aren't well-known seems to me to be irrelevant: then it's Diamond's duty to tell us about them. The fact that they are weaker makes the omission a bit less glaring, but they should still be included if they are stronger than some of the pro-mechanisms that Diamond does mention.

I guess I have the intuition that it is not very honest to fail to present contra-mechanisms that are stronger than some of the pro-mechanisms. But you don't share that intuition?

Thanks, this is useful. We may compare with policy debates. The reason any individual's arguments in a policy debate might either be that they are biased or that they are intentionally only putting forward arguments that support their position for strategic reasons. It could be argued that there is a convention to the effect that this is allowable, particularly in e.g. political debates.

Similarly, the reason why a multiple factor explanation is one-sided might either be bias or that the author intentionally leaves out mechanism playing in the other direction for strategic reasons. (Thanks for reminding me of this!) In your opinion, there is a convention to the effect that this is allowable. I'm a bit more unsure about this, but it is always hard to establish what the implicit conventions regulating various sort of behaviour are, particularly complex and abstract sorts of behaviour as this. Also, if there is such a convention, it could be argued that it arose precisely because people are normally confirmation biased, which has led them to regularly give these sorts of one-sided explanations, which in turn made this a convention.

I do not know whether Diamond is biased or if he is doing this intentionally, but would guess at the former. I think, firstly, that he should be much clearer over what he is doing. If he only wants to list mechanisms playing in the one direction, they should explicitly say so.

But I also think it is unreasonable to only list mechanisms playing in the one direction, especially when there are stronger mechanisms playing in the other direction. Learning about those other mechanisms is clearly very useful for the reader wanting to get a grasp of why the agricultural revolution first occurred in the Fertile Crescent.

Our intuitions are a bit muddled here, I think, because it is so hard to obtain reliable knowledge concerning why the agricultural revolution occurred in the Fertile Crescent. Let us therefore instead look at an example where we do have somewhat better knowledge: climate change. When explaining why the climate has got hotter, it would be unreasonable, at least in a serious scientific context, not to mention mechanisms that have forced the climate to grow colder than it otherwise would have, such as global dimming. Indeed, the Wikipedia article on global warming does mention global dimming. Mentioning this factor is very important not the least because it tells us that the mechanisms warming the Earth are stronger than we would have had reason to believe if there hadn't been any counteracting factors.

One important upshot of this is that the notion of an "explanation" can be a bit misleading here. We need to be very clear over what it is that we want to explain. It is true that people sometimes do understand "Why did Athens beat Sparta?" as a call to list all factors that played to Athens advantage, and none that played to Sparta's advantage. But under normal circumstances, that is not what we should be interested in.

The downside is that given our psychology as it is, I suspect we think about things better when people are creating hypotheses and arguing for/against them rather than contesting the detail of a list of possible factors with no clear conclusion.

I think that you could come with a clear conclusion even if you mention counteracting forces. Indeed, I think that Diamond's theory is basically right and that he could show that these counteracting forces were clearly too weak to overcome the Fertile Crescent's geographical advantages (though this is just a hunch - I'm not an expert on that issue and only interested in this as an example of the methodological problem under discussion).

Very interesting. I wonder how general the roles are. What you talk about at the end is basically bystander effect: I believe that different people are more or less vulnerable to that, and I wonder whether being more 'bystander' prone goes with being more likely to go along with pressure to conform (Millgram etc.) and possibly (to make it clear this isn't a straightforwardly ethical thing) more likely to collaborate in Prisoner's Dilemma. The most important role question might be simply whether you see yourself as a generalised Agent with responsibility for what actually happens beyond fulfilling set roles you've been given.

To quote HPMOR again: 'PC or NPC, that is the question'

Thanks, good comment. Yes Diamond wants to give a compelling alternative to 'innate cultural/genetic superiority'. When he is doing that, it is, however, his responsibility to discuss evidence that tells against his theory too, such as geographical factors decreasing the chance that the agricultural revolution would occur in the Fertile Crescent. What he should have said is that yes, there are such factors, but that the Fertile Crescent still had, all things considered, geographical advantages.

It is true that when explaining "Why did Athens beat Sparta?" we don't focus on Sparta's advantages over Athens. I am however to a certain extent questioning that practice which I think comes from our System 1-driven urge for one-sided stories. It depends a bit on context, but normally we should be most interested in learning about the factors that had most causal impact on the event in question. It should be more valuable to learn of a factor that played strongly to Sparta's advantage than one that played weakly to Athens' advantage.

In a way what we want to explain is not "Why did Athens beat Sparta?" but rather "Why did Athens beat Sparta with amount x?" since we know the latter. Now with the latter formulation, it becomes clear that unless x is very large (whatever that means) some of the factors used to answer this question should play to Sparta's advantage.

This is very interesting indeed! I'm not sure how much we can get to bias, or whether it's about what the argument is trying to say. Is he asserting that those 8 are the (only) relevant things that could make agriculture more likely? It's awhile since I read it, but I saw it more as saying that those 8 are the reason why historically it was the Fertile Crescent. Not that it would always be those things on any remotely similar world, or even necessarily that it would always be there if you re-ran history. In fact, as you say, he seems to mostly be arguing why it's plausibly NOT the 'people from the Fertile Crescent are superior' argument. Or more strongly, why the geographical case is more compelling than the gene-based one.

Say there's a ninth category (I dunno, 'distance from steppes which tend to be full of dangerous nomads') which Fertile Crescent scores badly on, and which makes it 'less likely' to develop agriculture. If what we're trying to explain is why Fertile Crescent succeeded, we don't necessarily focus on that. If we wanted to give a complete explanation, we might do, but it's not necessary. Similarly, if we wanted to say 'why did Sparta beat Athens' we could point to the army, and if we wanted to ask 'why did Athens beat Sparta', we'd point to the navy (or whatever). The fact we can go either way shows that this explanation isn't strong enough to be predictive, but it gives a compelling alternative to 'innate cultural/genetic superiority'