Sorry for not replying earlier, I have lots of stuff going on right now.
So basically you are describing a scenario where humanity goes digital via WBE and lives inside a utopian virtual reality (either entirely or most of the time) thus solving all or most of our current problems without the need to create FAI and risk utter destruction in the process. Not a bad question to have asked as far as I am concerned - it is a scenario I considered as well.
However, there are many good objections to this virtual reality idea, but I'll just name the most obvious that comes to mind:
If anyone but a FAI is running this utopia, it seems very plausible that it would result in a dystopia, or at best something vaguely resembling our current world. Without some kind of superior intelligence monitoring the virtual reality, mere humans would have to make decisions about how Utopiaworld is run. This human fiddling makes it vulnerable to all kinds of human stupidities. Now, if the timeline this "solution" is supposed to work approaches near-infinity, then at some point collapse due to human error seems inevitable. Moreover, a virtual world opens up possibilities (and problems!) we can hardly imagine right now - possibilities and problems vast enough, that a human mind may simply not be adequate in order to solve and control them.
So unless you are willing to change the human mind in order to adapt it to a utopian virtual reality, it seems to be no viable option. Why mandatory brain-rewiring before uploading a human mind into a virtual reality run by human-like agents would be grotesque I won't elaborate further.
I guess what my objection really comes down to is that humans are stupid, and thus we cannot be trusted to do anything right in the long run. Or at least we certainly can't do things right enough to build a lasting and well-operating utopia. For that to work we would need extensive brain enhancements and if you go down this path, I think you run into the exact same problems I described already:
We humans are self-obsessed shits and bolting on a rationality module won't change that. It would make us better at getting the things we want, but the things humans seem to want most of the time (judged by our current actions) are not working utopias but things typical for self-obsessed evolved agents: Power, social status and sex. Could this change by virtue of becoming more rational and intelligent? There's a good chance it would. People would certainly think more (and more efficiently) about the future once they are smarter, but if you make humans sufficiently smart to produce something like a virtual utopia, you have already taken the first step to creating the uFAI you hoped to avoid in the first place. Humans would want to become even smarter than they (and all those other enhanced humans) already are, thus starting a race to the top. The result may be something terrible that has been endlessly envisioned throughout the ages: Powerful gods with the petty goals of humans.
So my current conclusion after thinking about the virtual reality scenario is that it's not a viable long-term solution for humanity. That is not to say that I've extensively studied and thought about this option, but I think the objection I detailed is pretty convincing.
In a sentence: We're not smart enough to directly construct a utopia (whether real or virtual) and if we actually made ourselves smart enough to do that, we'd probably raise the odds for uFAI. So risk-wise we would be worse off than trying to build a FAI from scratch, which would be more competent at solving our problems than humans could ever be (whether intelligence-enhanced or not).
Also, I realize that a virtual reality is a great way to absolve humanity of resource scarcity and restricted bodily abilities/requirements, but virtually all other problems (moral, social etc.) are left rather untouched by the virtual reality solution. A virtual reality is not automatically a utopia. In a way it would be like living in a world where everyone is immortal and filthy rich - everyone has huge power over material (or as in our case virtual) "things", yet this doesn't solve all your social or personal problems.
Thank you for taking the time to write out such a thoughtful reply. I will be taking the time to return the favor shortly.
EDIT: Here's the (long) reply:
If anyone but a FAI is running this utopia
What does it mean to run a utopia? In order to run something, one must make decisions. What sort of decisions would this person or FAI be making? I realize it's hard to predict what exact scenario it would be like, but we can speculate all we like (and then see which one's sound most realistic/probable). Also, who said that anyONE would be "running" t...
If you solve the problem of human-friendly self-improving AI, you have indirectly solved every problem. After spending a decent amount of time on LW, I have been convinced of this premise and now I would like to devote my life to that cause.
Currently I'm living in Germany and I'm studying psychology in the first semester. The university I'm studying at has a great reputation (even internationally if I can believe the rankings) for the quality of its scientific psychology research and it ranks about #2nd or #3rd place when it comes to various psy-science-related criteria out of about 55 German universities where one can study psychology. Five semesters of statistics in my Bachelor of Science might also hint at that.
I want to finish my Bachelor of Science and then move on to my Master, so in about 5 years I might hit my "phase of actual productivity" in the working field. I'm flirting with cognitive neuroscience, but haven't made my decision yet - however, I am pretty sure that I want to move towards research and a scientific career rather than one in a therapeutic field.
Before discovering lesswrong my most dominant personal interest in psychology has been in the field of "positive psychology" or plainly speaking the "what makes humans happy" field. This interest hasn't really changed through the discovery of LW, as much as it has evolved into: "how can we distill what makes human life worthwhile and put it into terms a machine could execute for our benefit"?
As the title suggests, I'm writing all this because I want some creative input from you in order to expand my sense of possibilities concerning how I can help the development of friendly AI from the field of psychology most effectively.
To give you a better idea of what might fit me, a bit more background-info about myself and my abilities seems in order:
I like talking and writing a lot, mathematically I am a loser (whether due to early disgust or incompetence I can't really tell). I value and enjoy human contact and have constantly moved from being an introvert towards being an extrovert by several cognitive developments I can only speculate on. I would probably easily rank in the middle field of any positive extroversion scale nowadays. My IQ seems to be around 134 if one can trust the "International High IQ Society" (www.highiqsociety.org), but as mentioned my abilities probably lie more in the linguistic and to some extent analytic sphere than the mathematical. I understand Bayes' Theorem but haven't read the quantum mechanics sequence and many "higher" concepts here are still above my current level of comprehension. Although I haven't tried all that hard yet to be fair.
I have programmed some primitive HTML and CSS once and didn't really like it. From that experiecne and my mathematical inability I take away, that programming wouldn't be the way that I could contribute most efficiently towards friendly AI-research. It is none of my strenghts or at least it would take a lot of time to develop that, which would probably be better spent somewhere else. Also I quite surely wouldn't enjoy it as much as work in the psychological realm with humans.
My English is almost indistinguishable from that of a native speaker and I largely lack that (rightfully) despised and annoying German accent, so I could definitely see myself giving competent talks in English.
Like many of you I have serious problems with akrasia (regardless of whether that's a rationalist phenomenon or whether we are just more aware of it and tend to do types of work that tempt it more readily). Before I learned of how to effectively combat it (thank you Piers Steel!), I had plenty of motivation to get rid of it and sunk insane efforts into overcoming it, although ultimately it was largely an unsuccessful undertaking due to half-assed pop-science and the lack of a real insight about what procrastination is caused by and how it actually functions. Now that I know how to fix procrastination (or rather now that I know that it can't be fixed, as much as it has to be managed in a similar fashion to any given drug-addition), my motivation to overcome it is almost gone and I feel myself slacking. Also, the high certainty that there is no such thing as "free will" may have played a serious part in my procrastination habits (interestingly, there are at least two papers I recall showing this correlation). In a nutshell: Procrastination is a problem that I need to address, since it is definitely the Achilles' heel of my performance and it's absolutely crippling my potential. I probably rank middle-high on the impulsiveness- (and thus also on the procrastination-) scale.
That should be an adequate characterization of myself for now.
I am absolutely open for suggestions that are not related to the neuroscience of "what makes humans happy and how do I distill those goals and feelings into something a machine could work with"-field, but currently I am definitely flirting with that idea, even though I have absolutely no clue how the heck this area of research could be sufficiently financed in a decade from now and how it could spit out findings precise enough to benefit the creation of FAI. Yet maybe it's just a lack of imagination.
Trying to help set up and evolve a rationalist community in Germany would also be a decent task, but compared to specific research that actually directly aids our goals... I somehow feel it is less than what I could reasonably achieve if I really set my mind to it.
So tell me, where does a German psychologist go nowadays to achieve the biggest possible positive impact in the field of friendly AI?