Will_Newsome comments on Muehlhauser-Goertzel Dialogue, Part 1 - Less Wrong

28 Post author: lukeprog 16 March 2012 05:12PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (161)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 17 March 2012 03:08:50PM 4 points [-]

There is no formal mapping from the mugging to AGI existential risk.

Comment author: Will_Newsome 19 March 2012 02:42:51AM 3 points [-]

Not sure what you mean. It seems clear that both mugging and x-risk have a Pascalian flavor. ('Course, I personally think that the original wager wasn't fallacious...)

Comment author: gwern 19 March 2012 03:12:27AM 2 points [-]

The mugging has a referential aspect to it, referring either to your elicited prior or your universal prior. Whatever probability you give or whatever the universal is, the mugger solves for x in the obvious inequality (x times probability > 1) and claims the mugging yields x utility.

(Not that it matters, but my own belief is that the solution is to have a prior that takes into account the magnitude, avoiding the complexity issue by bounding the loss and identifying this loss with the prefix you need in the Kolmogorov setting.)

In contrast, there is no such referentiality in assessing AGI existential risk that I've heard of. FAIs don't offer to bootstrap themselves into existence with greater probability or... something like that. I'm really not sure how one would construct an x-risk argument isomorphic

(Possibly XiXi experienced a brainfart and meant the more common accusation that the general argument for existential risk prevention is Pascal's wager with finite stakes and more plausible premises.)

Comment author: Will_Newsome 19 March 2012 03:36:25AM 3 points [-]

Ah, I was assuming that everyone was assuming that XiXiDu had had a brainfart.