Why do some otherwise very smart people fall for this sleight of hand? I think it is because it panders to their narcissism. To regard oneself as one of a select few far-sighted thinkers who might turn out to be the saviors of mankind must be very rewarding.

I think this is a bad line of thought even before we get to the hypothesis that people are pushing UFAI risks for the money.

For one thing, people just get things wrong a lot-- it doesn't take bad motivations.

For another, it's very easy to jump to the conclusion that what seems to be correct to you is so obviously correct that other people must be getting it wrong on purpose.

For a third, even if you're right that other people are engaged in motivated thinking, you might be wrong about the motivation. For example, concern about UFAI might be driven by anxiety, or by "ooh, shiny! cool idea!" more than by narcissism or money.

advancedatheist, how sure are you of your motivations?