timtyler comments on A taxonomy of Oracle AIs - Less Wrong

13 Post author: lukeprog 08 March 2012 11:14PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (51)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 09 March 2012 05:28:15PM *  2 points [-]

The goal of the predictor is to make predictions that come true.

It isn't clear that this applies to prediction systems like Solomonoff induction or Levin search - which superficially do not appear to be goal-directed.

This will lead to the predictor behaving somewhat like the Oracle in Greek mythology, making whatever prophecy is maximally self-fulfilling, sometimes with disastrous results.

I found an example:

The oracle delivered to Oedipus what is often called a "self-fulfilling prophecy", in that the prophecy itself sets in motion events that conclude with its own fulfilment.

It is also discussed here.