Nick_Tarleton comments on The Costs of Rationality - Less Wrong

32 Post author: RobinHanson 03 March 2009 06:13PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (78)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nick_Tarleton 05 March 2009 06:58:28PM *  1 point [-]

"Not to my benefit" is ambiguous; I assume you mean working against other goals, like happiness or other people not dying. But since optimizing for one thing means not optimizing for others, every goal has this property relative to every other (for an ideal agent). Still, the concept seems very useful; any thoughts on how to formalize it?

Comment author: swestrup 05 March 2009 09:00:54PM 1 point [-]

I don't really have any ideas other than the "negative net sum" worth I mentioned above, but then that just begs the question of what metric one is using to measure worth.