The way I see it, even if we completely solve decision theory, there are so many other problems involved with building an FAI that the success probability (unless we first develop WBE or intelligence amplification) is still tiny. So working on decision theory is counterproductive if it raises the probability of UFAI coming before WBE/IA by even a small delta.
I don't think it's possible to separate decision theory from FAI theory, and if we all stop developing FAI theory, we lose automatically.
Of course I'm not suggesting we stop such work permanently, only until WBE or IA arrives. (Or when they are close enough that the probability of UFAI coming first is no longer significant, or if it becomes clear that de novo AI is much easier than WBE and IA and we have no choice but to push for FAI directly.)
What changes with WBE? Waiting for it already forsakes significant part of the expected future, but then what? It accelerates the physical timeline towards FAI and catastrophe both, and squeezing more FAI than catastrophe out of it (compared to pre-WBE ratio) requires rather unlikely circumstances.
The way I see it, even if we completely solve decision theory, there are so many other problems involved with building an FAI that the success probability (unless we first develop WBE or intelligence amplification) is still tiny.
Yes. (With the caveat that I'm...
Here is a short new publication from the Singularity Institute, on the 2-day workshop that followed Singularity Summit 2011.
Note the new publication design. We are currently porting our earlier publications to this template, too.