You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Kaj_Sotala comments on AI Risk and Opportunity: A Strategic Analysis - Less Wrong Discussion

8 Post author: lukeprog 04 March 2012 06:06AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (161)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 29 March 2012 10:12:01PM *  3 points [-]

If we assume some large probability that we end up deciding not to have an endgame at all (i.e., not to try to actually build FAI with unenhanced humans), then it's no longer clear "the time hasn't come yet".

This is something we'll know better further down the road, so as long as it's possible to defer this decision (i.e. while the downside is not too great, however that should be estimated), it's the right thing to do. I still can't rule out that there might be a preference definition procedure (that refers to humans) simple enough to be implemented pre-WBE, and decision theory seems to be an attack on this possibility (clarifying why this is naive, for example, in which case it'll also serve as an argument to the powerful in the WBE race).

The "current state of the theory" seems to have little to do with it. (Edit: No that's too strong. Let me amend it to "one consideration among many".)

Well, maybe not specifically current, but what can be expected eventually, for the closed project to benefit from, which does seem to me like a major consideration in the possibility of its success.