One issue I've mentioned before and I think is worth addressing is how much of the ability to quickly self-improve might depend on strict computational limits from theoretical computer science (especially complexity theory). If P != NP in a strong sense then recursive self-improvement many be very difficult.

More explicitly, many problems that are relevant for recursive self-improvement (circuit design and memory management for example) explicitly involve graph coloring and traveling salesman variants which are NP-hard or NP-complete. In that context, it could well be that designing new hardware and software will quickly hit diminishing marginal returns. If P, NP. coNP, PSPACE, and EXP are all distinct in a strong sense, then this sort of result is plausible.

There are problems with this sort of issue. One major one is that the standard distinctions between complexity classes are all in terms of Big-Os. So it could well be that the various classes are distinct but that the constants are small enough such that for all practical purposes one can do whatever one wants. There are also possible loopholes. For example, Scott Aaronson has shown that if one has access to closed time-like curves then one can quickly solve any problem in PSPACE. This is one of the more exotic loopholes. Quantum computers also may have more efficient computation. At this point, while almost everyone believes that P != NP, the notion that BQP doesn't contain NP seems substantially more uncertain. Finally, and most mundanely, it may be that even as the general problems are tough, the average cases of NP problems may not be difficult, and the specific examples that actually come up may have enough regularity that they can be solved more efficiently. Indeed, in one sense, part of why Deolalikar's attempted proof that P !=NP failed is that statistically, NP looks easy on average (more particularly, k-SAT is NP-complete for k >=3, but k-SAT for general k looks statistically a lot like 2-SAT).

It would seem to me that at this point that a lot more attention should be paid to computational complexity and what it has to say about the plausibility of quick recursive self-improvement.