JohnSidles comments on Cambridge Meetup: Talk by Eliezer Yudkowsky: Recursion in rational agents - Less Wrong

7 Post author: jimrandomh 15 October 2013 04:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (5)

You are viewing a single comment's thread. Show more comments above.

Comment author: JohnSidles 15 October 2013 09:37:05PM 0 points [-]

A preprint would be terrific too.

A tough(?) question and a tougher(?) question: When self-modifying AI's are citizens of Terry Tao's Island of the Blue-Eyed People/AIs, can the AIs trust one another to keep the customs of the Island? On this same AI-island, when the AI's play the Newcomb's Paradox Game, according to the rules of balanced advantage, can the PredictorAIs outwit the ChooserAIs, and still satisfy the island's ProctorAIs?

Questions in this class are tough (as they seem to me), and it is good to see that they are being creatively formalized.