= 27d567bc2d48d9f40e9ccc20ea370b15)
TobyBartels comments on Harry Potter and the Methods of Rationality discussion thread, part 3 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (560)
OK, this is off-topic, but why do people stop there? Why not ‘I always cooperate with agents I know will cooperate with me iff I cooperate iff they cooperate’, and so on? These are not equivalent.
Incidentally, in classical logic, I cooperate iff (you cooperate iff (I cooperate iff you cooperate)) is always true. (But we don't really have that here, because the modal operator ‘I know’ interferes.)
It isn't necessary to stop there, and you can follow that chain pretty much infinitely.
I think TDT jumps to the end of that regression by cooperating iff you and I are both implementations of the same abstract computation.
Yeah, but each stage is rather different from the one before; at no stage would you actually cooperate with yourself, since those ‘iff’s are so strict.
But if this (which I've seen here before) is not supposed to be what TDT really says, but just some handwaving to give the idea, then that's all right.
TDT hasn't been published in anything resembling a finished form, and I'm a curious amateur when it comes to decision theory, at best. I imagine there's more to it, but I can't really speculate about what it might be.
People stop there because going further starts hurting instead of helping. The PD payoff matrix implies that I want to avoid cooperating if I can, but it's more important that I get you to cooperate, even if in order to do that, I have to cooperate. Adding more restrictions on your reasons for cooperating can't make the outcome better for me, I only care that you do it.
Going one step further doesn't (generally) add restrictions; it just changes them. Consider:
Using classical logic after the modal operator, these reduce to:
Actually, now that I write it out like this, I can see why one would choose (3)!
It's important that there's an ‘if I know that’ instead of an ‘iff’, which I've seen before. But the version above is how I parsed WrongBot's statement, so hopefully WrongBot quoted it correctly. (The search function is not helping me find an original.)