I worry that this post seems very abstract.

The specific case I've made for "just build the damn FAI" does not revolve only around astronomical waste, but subtheses like:

• Stable goals in sufficiently advanced self-improving minds imply very strong path dependence on the point up to where the mind is sufficiently advanced, and no ability to correct mistakes beyond that point
• Friendly superintelligences negate other x-risks once developed
• CEV (more generally indirect normativity) implies that there exists a broad class of roughly equivalently-expectedly-good optimal states if we can pass a satisficing test (i.e., in our present state of uncertainty, we would expect something like CEV to be around as good as it gets, given our uncertainty on the details of goodness, assuming you can build a CEV-SI); there is not much gain from making FAI programmers marginally nicer people or giving them marginally better moral advice provided that they are satisficingly non-jerks who try to build an indirectly normative AI
• Very little path dependence of the far future on anything except the satisficing test of building a good-enough FAI, because a superintelligent singleton has enough power to correct any bad inertia going into that point
• The Fragility of Value thesis implies that value drops off very fast short of a CEV-style FAI (making the kinds of mistakes that people like to imagine leading to flawed-utopia story outcomes will actually just kill you instantly when blown up to a superintelligent scale) so there's not much point in trying to make things nicer underneath this threshold
• FAI is hard (relative to the nearly nonexistent quantity and quality of work that we've seen most current AGI people intending to put into it, or mainstream leaders anticipating a need for, or current agencies funding, FAI is technically far harder than that); so most of the x-risk comes from failure to solve the technical problem
• Trying to ensure that "Western democracies remain the most advanced and can build AI first" or "ensuring that evil corporations don't have so much power that they can influence AI-building" is missing the point (and a rather obvious attempt to map the problem into someone's favorite mundane political hobbyhorse) because goodness is not magically sneezed into the AI from well-intentioned builders, and favorite-good-guy-of-the-week is not making anything like a preliminary good-faith-effort to do high-quality work on technical FAI problems, and probably won't do so tomorrow either

You can make a case for MIRI with fewer requirements than that, but my model of the future is that it's just a pass-fail test on building indirectly normative stable self-improving AI, before any event occurs which permanently prevents anyone from building FAI (mostly self-improving UFAI (possibly neuromorphic) but also things like nanotechnological warfare). If you think that building FAI is a done deal because it's such an easy problem (or because likely builders are already guaranteed to be supercompetent), you'd focus on preventing nanotechnological warfare or something along those lines. To me it looks more like we're way behind on our dues.