= 27d567bc2d48d9f40e9ccc20ea370b15)
dankane comments on Timeless Decision Theory: Problems I Can't Solve - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (153)
Actually here's my argument why (ignoring the simulation arguments) you should actually refuse to give Omega money.
Here's what actually happened:
Omega flipped a fair coin. If it comes up heads the stated conversation happened. If it comes up tails and Omega predicts that you would have given him $1000, he steals $1000000 from you.
If you have a policy of paying you earn 10^6/4 - 10^3/4 -10^6/2 = -$250250. If you have a policy of not paying you get 0.
More realistically having a policy of paying Omega in such a situation could earn or lose you money if people interact with you based on a prediction of your policy, but there is no reason to suspect one over the other.
There's a similar problem with the Prisoner's Dilemma solution. If you formalize it as two of you are in a Prisoner's Dilemma and can see each other's code, then modifying your code to cooperate against the mirror matchup helps you in the mirror matchup, but hurts you if you are playing against a program that cooperates unless you would cooperate in a mirror matchup. Unless you have a reason to suspect that running into one is more likely than running into the other, you can't tell which would work better.
Having thought about it a little more, I think I have pinpointed my problem with building a decision theory in which real outcomes are allowed to depend on the outcomes of counterfactuals:
The output of your algorithm in a given situation will need to depend on your prior distribution and not just on your posterior distribution.
In CDT, your choice of actions depends only on the present state of the universe. Hence you can make your decision based solely on your posterior distribution on the present state of the universe.
If you need to deal with counterfactuals though, the output of your algorithm in a given situation should depend not only on the state of the universe in that situation, but on the probability that this situation appears in a relevant counterfactual and upon the results thereof. I cannot just consult my posterior and ask about the expected results of my actions. I also need to consult my prior and compute the probability that my payouts will depend on a counterfactual version of this situation.