In reading the Singularity Institute's research goals, and the ruminations of Yudkowski, Wei Dai, Nesov et al. in postings here, the approach to developing friendly AI which stands out the most, and from my perspective seems to just always have been the case, seems to be exclusively logic based in the vein of John McCarthy.
I am wondering how the decision was made to focus research for SIAI on the pure logic side, rather than, for example building a synthetic conscious which uses the brain as a model?
To be sure, nearly all AI approaches overlap at some point. However it seems a bit like reinventing the wheel from an almost purely theoretical perspective, when you have a very good model to work from.
Such a design would be harder to reason about.
Let's say you've got a prototype you want to improve. How do you tell if a proposed change would make it smarter, break it, introduce a subtle cognitive bias, or make the AI want to kill you?
In order to set on limits on the kinds of things an AI will do, you need to understand how it works. You can't be experimenting on a structure you partially understand, AND be certain that the experiments won't be fatal.
This is easier when you've got a clearly defined structure to the AI, and know how the parts interact, and why.
I declare this Open Thread open for discussion of Less Wrong topics that have not appeared in recent posts.