The claim that an organization is exceptionally well-suited to convert money into existential risk mitigation is an extraordinary one, and extraordinary claims require extraordinary evidence.

Reminder: I don't know if you were committing this particular error internally, but, at the least, the sentence is liable to cause the error externally, so: Large consequences != prior improbability. E.g. although global warming has very large consequences, and even implies that we should take large actions, it isn't improbable a priori that carbon dioxide should trap heat in the atmosphere - it's supposed to happen, according to standard physics. And so demanding strong evidence that global warming is anthropogenic is bad probability theory and decision theory. Expensive actions imply a high value of information, meaning that if we happen to have access to cheap, powerfully distinguishing evidence about global warming we should look at it; but if that evidence is not available, then we go from the default extrapolation from standard physics and make policy on that basis - not demand more powerful evidence on pain of doing nothing.

The claim that SIAI is currently best-suited to convert marginal dollars into FAI and/or general x-risk mitigation has large consequences. Likewise claims like "most possible self-improving AIs will kill you, although there's an accessible small space of good designs". This is not the same as saying that if the other facts of the world are what they appear at face value to be, these claims should require extraordinary evidence before we believe them.

Since reference class tennis is also a danger (i.e, if you want to conclude that a belief is false, you can always find a reference class in which to put it where most beliefs are false, e.g. classifying global warming as an "apocalyptic belief"), one more reliable standard to require before saying "Extraordinary claims require extraordinary evidence" is to ask what prior belief needs to be broken by the extraordinary evidence, and how well-supported that prior belief may be. Suppose global warming is real - what facet of existing scientific understanding would need to change? None, in fact; it is the absence of anthropogenic global warming that would imply change in our current beliefs, so that's what would require the extraordinary evidence to power it. In the same sense, an AI showing up as early as 2025, self-improving, and ending the world, doesn't make us say "What? Impossible!" with respect to any current well-supported scientific belief. And if SIAI manages to get together a pack of topnotch mathematicians and solve the FAI problem, it's not clear to me that you can pinpoint a currently-well-supported element of the world-model which gets broken.

The idea that the proposition contains too much burdensome detail - as opposed to an extraordinary element - would be a separate discussion. There are fewer details required than many strawman versions would have it; and often what seems like a specific detail is actually just an antiprediction, i.e., UFAI is not about a special utility function but about the whole class of non-Friendly utility functions. Nonetheless, if someone's thought processes were dominated by model risk, but they nonetheless actually cared about Earth's survival, and were generally sympathetic to SIAI even as they distrusted the specifics, it seems to me that they should support CFAR, part of whose rationale is explicitly the idea that Earth gets a log(number of rationalists) saving throw bonus on many different x-risks.