Why build SIAI?

For an important topic, it makes sense to have a dedicated research center. And in the end, SIAI is supposed to create a Friendly AI for real, not just to design it. As it turns out, SIAI also manages to serve many other purposes, like organizing the summits. As for FAI theory, I think it would have developed more slowly if Eliezer had apprenticed himself to a computer science department somewhere.

However, I do think we are at a point where the template of the existing FAI solution envisaged by SIAI could be imitated by mainstream institutions. That solution is, more or less, figure out the utility function implicitly supposed by the human decision process, figure out the utility function produced by reflective idealization of the natural utility function, create a self-enhancing AI with this second utility function. I think that is an approach to ethical AI which could easily become the consensus idea of what should be done.