What changes with WBE?
Did you read the part of the workshop report that talked about this?
Yes, and I agree, but it's not what I referred to. The essential part of the claim (as I accept it) is that given WBE, there exist scenarios where FAI can be developed much more reliably than in any feasible pre-WBE scenario. At the very least, dominating WBE theoretically allows to spend thousands of subjective years working on the problem, while in pre-WBE mode we have at most 150 and more likely about 50-80 years.
What I was talking about is probability of success. FAI (and FAI theory in particular, as a technology-independent component) is a race against AGI and other disasters, which become more likely as technology develops. In any given time interval, all else equal, completion of an AGI seems significantly more likely than of FAI. It's this probability of winning vs. losing the race in any given time that I don't see expected to change with the WBE transition. Just as FAI research gets more time, so is AGI research expected to get more time, unless somehow FAI researchers outrun everyone else for WBE resources, what I called "dominating WBE" above, but that's an unlikely feat, I don't have reasons for seeing that as more likely than just solving FAI pre-WBE.
In other words, we have two different low-probability events that bound success pre-WBE and post-WBE: solving the likely-too-difficult problem of FAI in a short time (pre-WBE), and outrunning "competing" AGI projects (post-WBE). If AGI is easy, pre-WBE is more important, because probability of surviving to post-WBE is then low. If AGI is hard, then FAI is hard too, and so we must rely on the post-WBE stage.
The gamble is on uncertainty about how hard FAI and AGI are. If they are very hard, we'll probably get to the WBE race. Otherwise, it's worth trying now, just in case it's possible to solve FAI earlier, or perhaps to develop the theory well enough to gain high-profile claim on dominating WBE and finishing the project before competing risks.
Just as FAI research gets more time, so is AGI research expected to get more time, unless somehow FAI researchers outrun everyone else for WBE resources, what I called "dominating WBE" above, but that's an unlikely feat, I don't have reasons for seeing that as more likely than just solving FAI pre-WBE.
In order for FAI to win pre-WBE, FAI has to get more resources than AGI (e.g., more, smarter researchers, computing power), but because FAI is much harder than AGI, it needs a large advantage. The "race for WBE" is better because it's a...
Here is a short new publication from the Singularity Institute, on the 2-day workshop that followed Singularity Summit 2011.
Note the new publication design. We are currently porting our earlier publications to this template, too.