But recursive self-improvement isn't always a recipe for FOOMing; diminishing returns can lead to an asymptotic bound, Zeno-style, even if technical refinements remain theoretically possible.
Yes. Early wins are very cheering and promising, but pretty much all growth curves follow a sigmoid. See the start? That's exponential. See the end? That's diminishing returns. This is one of those curves you see over and over.
So the trick is to find a new sigmoid to climb :-D
There's plenty of discussion about recursive self-improvement for AGIs, and self-improvement for ourselves, but I haven't come across (in memory) anything that specifically combines the two concepts. Arguably, increasing rationality skills would be recursive, but can anyone think of specific ways in which we can improve ourselves via iterative cycles? Is there a limit to how far we can currently improve our abilities by improving our abilities to improve our abilities? Or are these not the right questions; the concept a mere semantic illusion...
A place to start might be base-level anti-akrasia abilities. Are there physical steps we can do to force ourselves to self-improve when we otherwise lose the will or forget or move on to another interesting subject? Construct some sort of externally self-propelling regime that pulls us along a path to improvement? Could we "Incept" ourselves with the seed of a recursively self-improving paradigm?
-
Forgive the overbearingly inquisitive tone. This is my first post after lurking a couple years, and I guess even in the discussion section I subconsciously avoid making any potentially low-status claims of my own!