orthonormal comments on Yet another safe oracle AI proposal - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (33)
Style suggestion: give an informal overview of the idea, like your original comment, before going into the details. New readers need to see the basic idea before they'll be willing to wade into code.
Content suggestion: The main reason that I find your idea intriguing is something that you barely mention above: that because you're giving the AI an optimization target that only cares about its immediate progeny, it won't start cooperating with its later descendants (which would pretty clearly lead to un-boxing itself), nor upgrade to a decision theory that would cooperate further down the line. I think that part deserves more discussion.
Thanks, I've added a small overview section. I might edit this a little more later.