The main problem that remains unsolved is that somebody else can still build a fully self-improving UFAI. And that AI, being a lot smarter and more free than your limited AI, will crush it. The best way to prevent that from happening is to have a system that can monitor and understand pretty much everything, and that's pretty much FAI.
Is it really the case that an unrestricted growing entity will outcompete a restricted growing entity?
i.e. something like Cancer vs the Immune System.
I'm not convinced that the unrestricted growing entity will always win.
It seems to be a widely held belief around here that unfriendly artificial general intelligence is dangerous, and that (provably) friendly artificial general intelligence is the soundest counter to it.
But I'd like to see some analysis of alternatives. Here are some possible technical developments. Would any of these defuse the threat? How much would they help?
Are there other advances in computer science that might show up within the next twenty years, that would make friendly-AI much less interesting?
Would anything on this list be dangerous? Obviously, efficient algorithms for NP-complete problems would be very disruptive. Nearly all of modern cryptography would become irrelevant, for instance.