If you want to keep the search function from wireheading the world model then you have to code "don't break the world model" into your value function. This is a general contradiction to the Orthogonality Thesis. A sufficiently powerful world-optimizing artificial intelligence must have a value function that preserves the integrity of its world model, because otherwise it'll just wirehead itself, instead of optimizing the world.
If the value function says ~"maximise the number of paperclips, as counted by my paperclip-counting-machinery", a weak AI mig... (read more)
If the value function says ~"maximise the number of paperclips, as counted by my paperclip-counting-machinery", a weak AI mig... (read more)