AlexMennen comments on A taxonomy of Oracle AIs - Less Wrong

13 Post author: lukeprog 08 March 2012 11:14PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (51)

You are viewing a single comment's thread. Show more comments above.

Comment author: AlexMennen 09 March 2012 07:30:59PM *  0 points [-]

I don't think that would make much of a difference because then it still can answer conditional on what its answer would be when it is asked the same question again the next day, which would presumably be the same answer unless it got new information during that wait period.

A perhaps safer but less efficient alternative is to design a Predictor (without keeping any backup copies of its source code), such that with some small probability, it will delete its source code and shut down instead of answering your question. Then if you still want to know the answer, you have to design a new Predictor with the same specifications but a different algorithm. The Predictor's answer (if it gives one) refers to what would happen conditional on it shuts down.