Doug_S. comments on The Bedrock of Fairness - Less Wrong

25 Post author: Eliezer_Yudkowsky 03 July 2008 06:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (102)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Doug_S. 03 July 2008 10:20:53PM 0 points [-]

Certain desires help the human race, other desires hurt the human race, and these can be discovered in the same way we discover any other facts about the universe.

You simply passed the recursive buck to "help" and "hurt". I will let you take for granted the superintelligence's knowledge of, or well-calibrated probability distribution over, any empirical truth about consequences; but when it comes to the valuation of those consequences in terms of "helping" or "hurting" you must tell me how to compute it, or run a computation that computes how to compute it.

To rephrase the statement without passing the recursive buck:

Certain desires help the human race fulfill desires, other desires prevent the human race from fulfilling desires, and these can be discovered in the same way we discover any other facts about the universe.