I have to admit I've not taking the time to understand all your equations. But, I don't understand why adjusting the equations can have any effect on the wirehead problem. When you physically implement any of these solutions, the machine will always end up with a behavior selection system regulated by some goal measure. Whether you call that goal a reward, or the output of a utility function, doesn't change the fact, that the measure itself must be computed, and the result of that computation then determines the machine's selection of behaviors.
With such a system, the goal of the system is always, by definition, to find the behavior, which... (read 711 more words →)
I have to admit I've not taking the time to understand all your equations. But, I don't understand why adjusting the equations can have any effect on the wirehead problem. When you physically implement any of these solutions, the machine will always end up with a behavior selection system regulated by some goal measure. Whether you call that goal a reward, or the output of a utility function, doesn't change the fact, that the measure itself must be computed, and the result of that computation then determines the machine's selection of behaviors.
With such a system, the goal of the system is always, by definition, to find the behavior, which... (read 711 more words →)