It occurs to me that there is a roadblock for an AI to go foom; namely that it first has to solve the same "keep my goals constant while rewriting myself" problem that MIRI is trying to solve.
Otherwise the situation would be analogous to Gandhi being offered a pill that has a chance of making him into anti-Gandhi, and declining it.
If the superhuman - but not yet foomed! - AI is not yet orders of magnitude smarter than a hoo-mon, it may be a while before it is willing to power-up / go foom, since it would not want to jeapardize its utility function along the way.
Just because it can foom does not imply it'll want to foom (because of the above).
This is interesting, though I think it's less relevant for an entity made out of readable code. In the pill situation, if ghandi fully understood both his own biochemistry and the pill, all chance would be removed from the equation.
If it's worth saying, but not worth its own post, even in Discussion, it goes here.