I requested feedback about this paper here.
One of my conclusions was that you could, in theory, train a Solomonoff Induction-based reinforcement learning agent to produce arbitrary finite sequences of actions (non-self-destructive ones anyway) in response to specified sets of finite sense data - assuming you are allowed to program its reward function and give it fake memories dating back from before it was born.
This is essentially the same result as is claimed for O-Maximisers in the paper. This undermines the thesis that O-Maximisers somehow exhibit different dynamics from reinforcement learning agents.
Update on 2011-04-30: Bill Hibbard makes an almost identical point to the observations I made in this comment. You can see it in his post - on the AGI mailing list - here.
Response to Curt Welch:
Sadly, what he seems to have failed to realize, is that any actual implementation of an O-Maximizer or his Value-learners must also be reward maximizerr. Is he really that stupid so as not to understand they are all reward maximizer?
Zing! I guess he didn't think I was going to be reading that. To be fair, it may seem to him that I've made a stupid error, thinking that O-maximizers behave differently than reward maximizers. I'll try to explain why he's mistaken.
A reward maximizer acts so as to bring about universes in which the ...
Daniel Dewey, 'Learning What to Value'
Abstract: I.J. Good's theory of an "intelligence explosion" predicts that ultraintelligent agents will undergo a process of repeated self-improvement. In the wake of such an event, how well our values are fulfilled will depend on whether these ultraintelligent agents continue to act desirably and as intended. We examine several design approaches, based on AIXI, that could be used to create ultraintelligent agents. In each case, we analyze the design conditions required for a successful, well-behaved ultraintelligent agent to be created. Our main contribution is an examination of value-learners, agents that learn a utility function from experience. We conclude that the design conditions on value-learners are in some ways less demanding than those on other design approaches.