MrHen comments on Newcomb's Problem vs. One-Shot Prisoner's Dilemma - Less Wrong

12 Post author: Wei_Dai 07 April 2009 05:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (15)

You are viewing a single comment's thread. Show more comments above.

Comment author: MrHen 07 April 2009 06:11:32PM 1 point [-]

If you have good reason to beleive the superintelligence is a sucessful extrapolation of the values of it's creators [...]

This seems to abstract the problem so that you have two problems instead of one: is the SI a successful extrapolation and the validity of the creators' claims of their values. This seems less efficient unless one or both of these were already known to begin with.

Comment author: MichaelHoward 07 April 2009 10:18:09PM 1 point [-]

You don't need to trust the creators' claims - you're running their simulations, and you're damn good at understanding them and extrapolating the consequences because, well, you're superintelligent! Why would they even know they're simulated? They're just discussing one-shot PD on some blog.

As for the SI being a successful extrapolation, you run a few simulations of it's birth the same way, starting a few decades before. It's still cheaper and less messy than organizing a mutual reprogramming with the brain that's made of the next galaxy.

Then the problem largely reduces to:

  • Verifying the data you passed each-other about your births are accurate.

  • Verifying ethical treatment of each-others simulated creators - no "victory candescence" when you get your answer!