Vladimir_Nesov comments on Counterfactual Calculation and Observational Knowledge - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (183)
World-programs are a bad model for possible worlds. For all you know, there could be just one world-program (indeed you can consider an equivalent variant of the theory where it's so: just have that single world program enumerate all outputs of all possible programs). The element of UDT analogous to possible worlds is execution histories. And some execution histories easily indicate that 2+2=5 (if we take execution histories to be enumerations of logical theories, with world-programs axiomatic definitions of theories). Observations, other background facts, and your actions are all elements that specify (sets/events of) execution histories. Utility function is defined on execution histories (and it's usually defined on possible worlds). Probability given by mathematical intuition can be read as naming probability that given execution history (possible world) is an actual one.
So, you intended that the equivalence
be known to the agent's mathematical intuition? I didn't realize that, but my solution still applies without change. It just means that, as far as the agent's mathematical intuition is concerned, we have the following equivalences between predicates over sequences of execution histories:
is equivalent to
while
is equivalent to
Below, I give my guess at your UDT1.1 approach to the problem in the OP. If I'm right, then we use the UDT1.1 concepts differently, but the math amounts to just a rearrangement of terms. I see merits in each conceptual approach over the other. I haven't decided which one I like best.
At any rate, here is my guess at your formalization: We have one world-program. We consider the following one-place predicates over possible execution histories for this program: Given any execution history E,
CalculatorIsCorrect(E) asserts that, in E, the calculator gives the correct parity for Q.
"even"(E) asserts that, in E, the calculator says "even". Omega then appears to the agent and asks it what Omega should have written on the test sheet in an execution history in which (1) Omega blocks the agent from writing on the answer sheet and (2) the calculator says "odd".
"odd"(E) asserts that, in E, the calculator says "odd". Omega then (1) blocks the agent from writing on the test sheet and (2) computes what the agent would have said to Omega in an execution history F such that "even"(F). Omega then writes what the agent would say in F on the answer sheet in E.
Borrowing notation from my last comment, we make the following assumptions about the probability measures P_f. For all input-output maps f,
P_f(CalculatorIsCorrect) = 0.99,
P_f("even") = P_f("odd") = 1/2,
"even" and "odd" are uncorrelated with CalculatorIsCorrect under P_f.
The input-output maps to consider are
g: On seeing "even", write "even" and tell Omega, "Write 'even'."
h: On seeing "even", write "even" and tell Omega, "Write 'odd'."
The utility U(E) of an execution history E is 1 if the answer on the sheet in E is the true parity of Q. Otherwise, U(E) = 0.
The expected payoffs of g and h are then, respectively,
EU(g) = P_g("even" & CalculatorIsCorrect) * 1 + P_g("even" & ~CalculatorIsCorrect) * 0 + P_g("odd" & CalculatorIsCorrect) * 0 + P_g("odd" & ~CalculatorIsCorrect) * 1 = 1/2 * 0.99 * 1 + 1/2 * 0.01 * 1 = 0.50.
EU(h) = P_h("even" & CalculatorIsCorrect) * 1 + P_h("even" & ~CalculatorIsCorrect) * 0 + P_h("odd" & CalculatorIsCorrect) * 1 + P_h("odd" & ~CalculatorIsCorrect) * 0 = 1/2 * 0.99 * 1 + 1/2 * 0.99 * 1 = 0.99.