amcknight comments on Thoughts and problems with Eliezer's measure of optimization power - Less Wrong

17 Post author: Stuart_Armstrong 08 June 2012 09:44AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (23)

You are viewing a single comment's thread.

Comment author: amcknight 08 June 2012 08:59:39PM 0 points [-]

An additional problem with the original definition of optimization power is that it requires the agent to have a rich set of preferences. If it doesn't prefer much but would have been very effective with its capacities had it had more preferences, I'd still say the optimization process was powerful. It seems to me that the most useful concept of optimization power will be independent of the optimizer's actual preferences (or utility).