"What is missing for the SIAI to actually start working on friendly AI?"
I think that question is answered by Yudkowsky in his interview with Baez:
"I probably need to take at least a year to study up on math, and then—though it may be an idealistic dream—I intend to plunge into the decision theory of self-modifying decision systems and never look back. (And finish the decision theory and implement it and run the AI, at which point, if all goes well, we Win.)"
Yudkowsky's position, widely known, is that it is unsafe to do otherwise. I imagine that is why they are not funding researchers to work on extending MOSES (or any other AGI work for that matter), but that's just speculation on my part.
To learn more about the work people are doing to build AGI, check out the conferences series on AGI at http://agi-conf.org/ agi-conf.org, organized by Ben Goertzel, advisor to SIAI (formerly Director of Research). Videos of most of the talks and tutorials are available for free, along with PDFs of the conference papers.
"What is missing for the SIAI to actually start working on friendly AI?"
The biggest problem in designing FAI is that nobody knows how to build AI. If you don't know how to build an AI, it's hard to figure out how to make it friendly. It's like thinking about how to make a computer play chess well before anybody knows how to make a computer.
In the meantime, there's lots of pre-FAI work to be done. There are many unsolved problems in metaethics, decision theory, anthropics, cosmology, and other subjects that seem to be highly relevant to later F...
One of the reasons that I am skeptical of contributing money to the SIAI is that I simply don't know what they would do with more money. The SIAI currently seems to be viable. Another reason is that I believe that an empirical approach is required, that we need to learn more about the nature of intelligence before we can even attempt to solve something like friendly AI.
I bring this up because I just came across an old post (2007) on the SIAI blog:
Some questions:
I also have some questions regarding the hiring of experts. Is there a way to figure out what exactly the current crew is working on in terms of friendly AI research? Peter de Blanc seems to be the only person who has done some actual work related to artificial intelligence.
I am aware that preparatory groundwork has to be done and capital has to be raised. But why is there no timeline? Why is there no progress report? What is missing for the SIAI to actually start working on friendly AI? The Singularity Institute is 10 years old, what is planned for the decade ahead?