Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

RobinZ comments on Value Uncertainty and the Singleton Scenario - Less Wrong

8 Post author: Wei_Dai 24 January 2010 05:03AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (28)

You are viewing a single comment's thread. Show more comments above.

Comment author: RobinZ 28 January 2010 03:51:03PM 0 points [-]

Is it possible to enforce that? It seems like specifying a bottom line to me.

Comment author: Peter_de_Blanc 28 January 2010 07:37:21PM 3 points [-]

It would be specifying a bottom line if each sub-agent could look at any result and say afterwards that this result supports its position. That's not what I'm suggesting. I'm saying that each sub-agent should make predictions as if its own value system is correct, rather than having each sub-agent use the same set of predictions generated by the super-agent.

Comment author: RobinZ 28 January 2010 08:01:44PM 0 points [-]

Quick dive into the concrete: I think that legalization of marijuana would be a good thing ... but that evaluation is based on my current state of knowledge, including several places where my knowledge is ambiguous. By Baye's Rule, I can't possibly have a nonzero expectation for the change in my evaluation based on the discovery of new data.

Am I misunderstanding the situation you hypothesize?

Comment author: Peter_de_Blanc 28 January 2010 10:39:13PM 1 point [-]

You can have a nonzero expectation for the change in someone else's evaluation, which is what I was talking about. The super-agent and the sub-agent have different beliefs.

Comment author: RobinZ 29 January 2010 12:17:41AM 0 points [-]

I see - that is sensible.