Wei_Dai comments on Singularity Summit 2011 Workshop Report - Less Wrong

7 Post author: lukeprog 01 March 2012 06:12AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (30)

You are viewing a single comment's thread.

Comment author: Wei_Dai 01 March 2012 09:18:11PM 5 points [-]

What is the value of strategy vs. object-level progress toward a positive Singularity?

I think strategy is currently being undervalued, although this workshop and other recent SIAI work is alleviating that concern quite a bit. I'm still not sure why there is so little discussion of Singularity strategies on LW, compared to say decision theory, CEV, or Oracle AI, even as SIAI has started doing a lot more work in that area. Is it that most LW posts can be considered replies/followups to Eliezer's posts, and he never wrote one about Singularity strategies?

Comment author: lukeprog 01 March 2012 10:16:03PM *  7 points [-]

Almost none of the strategy work has been published, but we've had thousands upon thousands of hours of dialogue between SIAI and FHI researchers and others like Paul Christiano (who is now a research associate) and Owain Evans. Carl has been in Oxford the last two weeks working with FHI folk on strategy issues almost every day. SIAI had a series of 10-hour strategy meetings in January. Believe me, lots of strategy work is being done, but I understand that you're not seeing hardly any of it. Much of the strategy work that's been done by SIAI and FHI in the past five years is, thankfully, ending up in Bostrom's forthcoming monograph on machine superintelligence.