= 27d567bc2d48d9f40e9ccc20ea370b15)
Nisan comments on A taxonomy of Oracle AIs - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (51)
This is the first time I've seen this Predictor argument. It would be nice to make a more formal version of this argument (maybe a theorem that certain Predictors become certain kinds of expected-utility maximizers), but doing so would require some formalization of logical uncertainty.
This sort of Predictor is essentially computing a fixed point of a world function that takes the prediction as input and outputs a probability distribution over outcomes conditional on the Predictor making this prediction. If the world function is independent of the prediction, than this works normally, and there is one fixed point where you want it to be.
This raises the question of what the Predictor would do when the world function has multiple fixed points (e.g. U(p) = p), or no fixed points (e.g. U(p) = 1 if p<.5 else 0).