Comment author:Wei_Dai
03 October 2012 01:03:44PM
1 point
[-]

I think Yudkowsky is a Platonist, and I'm not sure he has a consistent position on modal realism, since when arguing on morality he seemed to espouse it: see his comment here.

Thanks for the link. That does seem inconsistent.

I don't think that "You should two-box in Newcomb's problem." is actually a normative statement, even if it contains a "should": you can rephrase it epistemically as "If you two-box in Newcomb's problem then you will maximize your expected utility".

This comment should help you understand why I disagree. Does it make sense?

Comment author:V_V
03 October 2012 03:01:02PM
2 points
[-]

This comment should help you understand why I disagree. Does it make sense?

I don't claim that all injunctions can be rephrased as epistemic statements. I claim that decision-theoretic injunctions can be rephrased as epistemic statements. Moral injunctions can't.

Comment author:Wei_Dai
03 October 2012 09:30:00PM
*
0 points
[-]

I don't claim that all injunctions can be rephrased as epistemic statements. I claim that decision-theoretic injunctions can be rephrased as epistemic statements. Moral injunctions can't.

I'm confused by your reply because the comment I linked to tried to explain why I don't think "You should two-box in Newcomb's problem" can be rephrased as an epistemic statement (as you claimed earlier). Did you read it, and if so, can you explain why you disagree with its reasoning?

ETA: Sorry, I didn't notice your comment in the other subthread where you gave your definitions of "decision-theoretic" vs "moral" injunctions. Your reply makes more sense with those definitions in mind, but I think it shows that the comment I linked to didn't get my point across. So I'll try it again here. You said earlier:

I don't think that "You should two-box in Newcomb's problem." is actually a normative statement, even if it contains a "should": you can rephrase it epistemically as "If you two-box in Newcomb's problem then you will maximize your expected utility".

A causal decision theorist (C) and an evidential decision theorist (E) have different definitions of "maximize your expected utility", and so when C says to E "you should two-box in Newcomb's problem" he is not just saying "If you two-box in Newcomb's problem then you will maximize your expected utility according to the CDT formula" since E wouldn't care about that. So my point is that "you should two-box in Newcomb's problem" is usually not a "decision-theoretical injunction" in your sense of the phrase, but rather a normative statement as I claimed.

Comment author:V_V
04 October 2012 12:07:59PM
*
0 points
[-]

A causal decision theorist (C) and an evidential decision theorist (E) have different definitions of "maximize your expected utility", and so when C says to E "you should two-box in Newcomb's problem" he is not just saying "If you two-box in Newcomb's problem then you will maximize your expected utility according to the CDT formula" since E wouldn't care about that. So my point is that "you should two-box in Newcomb's problem" is usually not a "decision-theoretical injunction" in your sense of the phrase, but rather a normative statement as I claimed.

I was assuming implicitely that we were talking in the context of EDT.

In general, you can say "Two-boxing in Newcomb's problem is the optimal action for you", where the definition of "optimal action" depends on the decision theory you use.

If you use EDT, then "optimal action" means "maximizes expected utility", hence the statement above is false (that is, it is inconsistent with the axioms of EDT and Newcomb's problem).

If you use CDT, then "optimal action" means "maximizes expected utility under a causality assumption". Hence the statement above is technically true, although not very useful, since the axioms that define Newcomb's problem specifically violate the causality assumption.

So, which decision theory should you use? An answer like "you should use the decision theory that determines the optimal action without any assumption that violates the problem constraints" seems irreducible to an epistemic statement. But is that actually correct?

If you are studing actual agents, then the point is moot, since these agents already have a decision theory (in practice it will be an approximation of either EDT or CDT, or something else), but what if you want to improve yourself, or build an artificial agent?

Then you evaluate the new decision theory according to the decision theory that you already have. Then, assuming that in principle your current decision theory can be described epistemically, you can say, for instance: "A decision theory that determines the optimal action without any assumption that violates the problem constraints is optimal for me".

If you want to suggest a decision theory to somebody who is not you, you can say: "A decision theory that determines the optimal action without any assumption that violates the problem constraints is optimal for you", or, more properly but less politely: "You using a decision theory that determines the optimal action without any assumption that violates the problem constraints are optimal for me".

Comment author:Wei_Dai
04 October 2012 11:12:26PM
*
2 points
[-]

Then you evaluate the new decision theory according to the decision theory that you already have.

I had similar thoughts before, but eventually changed my mind. Unfortunately it's hard to convince people that their solution to some problem isn't entirely satisfactory without having a better solution at hand. (For example, this post of mine pointing out a problem with using probability theory to deal with indexical uncertainty sat at 0 points for months before I made my UDT post which suggested a different solution.) So instead of trying harder to convince people now, I think I will instead try harder to figure out a better answer by myself (and others who already share my views).

## Comments (515)

BestThanks for the link. That does seem inconsistent.

This comment should help you understand why I disagree. Does it make sense?

I don't claim that all injunctions can be rephrased as epistemic statements. I claim that decision-theoretic injunctions can be rephrased as epistemic statements. Moral injunctions can't.

*0 points [-]I'm confused by your reply because the comment I linked to tried to explain why I don't think "You should two-box in Newcomb's problem" can be rephrased as an epistemic statement (as you claimed earlier). Did you read it, and if so, can you explain why you disagree with its reasoning?

ETA: Sorry, I didn't notice your comment in the other subthread where you gave your definitions of "decision-theoretic" vs "moral" injunctions. Your reply makes more sense with those definitions in mind, but I think it shows that the comment I linked to didn't get my point across. So I'll try it again here. You said earlier:

A causal decision theorist (C) and an evidential decision theorist (E) have different definitions of "maximize your expected utility", and so when C says to E "you should two-box in Newcomb's problem" he is not just saying "If you two-box in Newcomb's problem then you will maximize your expected utility

according to the CDT formula" since E wouldn't care about that. So my point is that "you should two-box in Newcomb's problem" isusuallynot a "decision-theoretical injunction" in your sense of the phrase, but rather a normative statement as I claimed.*0 points [-]I was assuming implicitely that we were talking in the context of EDT.

In general, you can say "Two-boxing in Newcomb's problem is the optimal action for you", where the definition of "optimal action" depends on the decision theory you use.

If you use EDT, then "optimal action" means "maximizes expected utility", hence the statement above is false (that is, it is inconsistent with the axioms of EDT and Newcomb's problem).

If you use CDT, then "optimal action" means "maximizes expected utility under a causality assumption". Hence the statement above is technically true, although not very useful, since the axioms that define Newcomb's problem specifically violate the causality assumption.

So, which decision theory should you use? An answer like "you should use the decision theory that determines the optimal action without any assumption that violates the problem constraints" seems irreducible to an epistemic statement. But is that actually correct?

If you are studing actual agents, then the point is moot, since these agents already have a decision theory (in practice it will be an approximation of either EDT or CDT, or something else), but what if you want to improve yourself, or build an artificial agent?

Then you evaluate the new decision theory according to the decision theory that you already have. Then, assuming that in principle your current decision theory can be described epistemically, you can say, for instance: "A decision theory that determines the optimal action without any assumption that violates the problem constraints is optimal for me".

If you want to suggest a decision theory to somebody who is not you, you can say: "A decision theory that determines the optimal action without any assumption that violates the problem constraints is optimal for you", or, more properly but less politely: "You using a decision theory that determines the optimal action without any assumption that violates the problem constraints are optimal for me".

*2 points [-]I had similar thoughts before, but eventually changed my mind. Unfortunately it's hard to convince people that their solution to some problem isn't entirely satisfactory without having a better solution at hand. (For example, this post of mine pointing out a problem with using probability theory to deal with indexical uncertainty sat at 0 points for months before I made my UDT post which suggested a different solution.) So instead of trying harder to convince people now, I think I will instead try harder to figure out a better answer by myself (and others who already share my views).