Comment author:JGWeissman
27 July 2009 06:03:30PM
1 point
[-]

Why are we concerned with the expected utility of some subset of the probability space? To find the expected utility of an action, you should sum over the products of the utility of the point with its conditional probability given that you take that action, over all points in the space. In effect, you are only considering actions that reduce the probability of some points to zero, and then renormalizes the probability of the remaining points.

Expected utility is usually written for actions, but it can be written as in the post as well, it's formally equivalent. This treatment of expected utility isn't novel in any way. Any action can be identified with a set of possibilities (outcomes) in which it happens. When you talk of actions that "don't reduce some probabilities to zero", you are actually talking about the effect of the actions on probability distributions of random variables, but behind those random variables is still a probability space on which any information is an element of sigma algebra, or a clear-cut set of possibilities.

Comment author:JGWeissman
27 July 2009 10:38:18PM
1 point
[-]

Expected utility is usually written for actions, but it can be written as in the post as well, it's formally equivalent.

How is it formally equivalent? How can I represent the expected utility of an action with arbitrary effects on conditional probability using the average, weighted by unconditional probabilities, of the utility of some subset of the possibilities, as in the post?

Let A be the action (set of possibilities consistent with taking the action), and O set of possible outcomes (each one rated by the utility function, assuming for simplicity that every concrete outcome is considered, not events-outcomes). We can assume . Then:

Trick question? P(A) is just a probability of some event, so depending on the problem it could be calculated in any of the possible ways. "A" can for example correspond to a value of some random variable in a (dynamic) graphical model, taking observations into account, so that its probability value is obtained from belief propagation.

Comment author:JGWeissman
27 July 2009 11:41:56PM
0 points
[-]

As I already explained, that only works for actions that exclude some outcomes and renormalize the probabilities of remaining outcomes, preserving the ratios of their probabilities.

Suppose O had 2 elements, x1 and x2, such that p(x1) = p(x2) = .5. If you take action A, then you have conditional probabilities p(x1|A) = .2 and p(x2|A) = .8. In this case, your transformation of P(x|A) = P(x, A)/P(A) does not work. Because A did not remove x1 as a possibility, it just made it less likely.

P(x|A) = P(x,A)/P(A) is by definition of conditional probability. You are trying to interpret x1 and x2 as events, while in grandparent comment x are elements of the sample space. If you want to consider non-concrete outcomes, compose them from smaller elements. For example, you can have P(O1)=P(O2)=.5, P(O1|A)=.2, P(O2|A)=.8, if O1={x1,x2}, O2={x3,x4}, A={x1,x3}, and p(x1)=.1, p(x2)=.4, p(x3)=.4, p(x4)=.1.

## Comments (35)

BestWhy are we concerned with the expected utility of some subset of the probability space? To find the expected utility of an action, you should sum over the products of the utility of the point with its conditional probability given that you take that action, over all points in the space. In effect, you are only considering actions that reduce the probability of some points to zero, and then renormalizes the probability of the remaining points.

Expected utility is usually written for actions, but it can be written as in the post as well, it's formally equivalent. This treatment of expected utility isn't novel in any way. Any action can be identified with a set of possibilities (outcomes) in which it happens. When you talk of actions that "don't reduce some probabilities to zero", you are actually talking about the effect of the actions on probability distributions of random variables, but behind those random variables is still a probability space on which any information is an element of sigma algebra, or a clear-cut set of possibilities.

How is it formally equivalent? How can I represent the expected utility of an action with arbitrary effects on conditional probability using the average, weighted by unconditional probabilities, of the utility of some subset of the possibilities, as in the post?

*1 point [-]Let A be the action (set of possibilities consistent with taking the action), and O set of possible outcomes (each one rated by the utility function, assuming for simplicity that every concrete outcome is considered, not events-outcomes). We can assume . Then:

How do you calculate P(A)?

Trick question? P(A) is just a probability of some event, so depending on the problem it could be calculated in any of the possible ways. "A" can for example correspond to a value of some random variable in a (dynamic) graphical model, taking observations into account, so that its probability value is obtained from belief propagation.

As I already explained, that only works for actions that exclude some outcomes and renormalize the probabilities of remaining outcomes, preserving the ratios of their probabilities.

Suppose O had 2 elements, x1 and x2, such that p(x1) = p(x2) = .5. If you take action A, then you have conditional probabilities p(x1|A) = .2 and p(x2|A) = .8. In this case, your transformation of P(x|A) = P(x, A)/P(A) does not work. Because A did not remove x1 as a possibility, it just made it less likely.

*0 points [-]P(x|A) = P(x,A)/P(A) is by definition of conditional probability. You are trying to interpret x1 and x2 as events, while in grandparent comment x are elements of the sample space. If you want to consider non-concrete outcomes, compose them from smaller elements. For example, you can have P(O1)=P(O2)=.5, P(O1|A)=.2, P(O2|A)=.8, if O1={x1,x2}, O2={x3,x4}, A={x1,x3}, and p(x1)=.1, p(x2)=.4, p(x3)=.4, p(x4)=.1.