HungryHobo comments on JFK was not assassinated: prior probability zero events - Less Wrong

20 Post author: Stuart_Armstrong 27 April 2016 11:47AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (47)

You are viewing a single comment's thread.

Comment author: HungryHobo 29 April 2016 04:14:50PM *  1 point [-]

Why must the oracle continue to believe it's messages weren't read?

In the example you give I'm guessing the reason you'd want an oracle to believe with cold certainty that it's messages won't be read is to avoid it trying to influence the world with them but that doesn't require that it continue to believe that later. As long as when it's composing and ouputing the message it believes solidly that it will never be read and nothing can move that belief from zero then that's fine. That does not preclude it being perfectly accepting that it's past messages were in fact read and basing it's beliefs about the world on that. That knowledge after all cannot shift the belief that this next message will never, ever ever be read unlike all the others.

Of course that brings up the question of why an oracle would even be designed as a goal based AI with any kind of utility function. Square peg, round hole and all that.

Comment author: Stuart_Armstrong 29 April 2016 04:48:26PM 0 points [-]

For Oracles, you can reset them after they've sent out their message. For autonomous AIs, this is more tricky.