PhilGoetz comments on Diplomacy as a Game Theory Laboratory - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (93)
Is there a link to an online explanation of this? When are the consequences of breaking an oath worse than a destroyed world? What did "world" mean when he said it? Humans? Earth? Humans on Earth? Energy in the Multiverse?
Given that you are still alive, posting, and connected within SIAI and LessWrong; and that they both probably expected that at the time; I don't think any such situation is possible. I think you're giving them shockingly little credit as rationalists, especially if you've read either Luminosity or Harry Potter and the Methods of Rationality.
Eliezer has proven, empirically, that his reputation is worth at least an amount on the order of the funding that SIAI has received to date - let's say half of it. Every time he opens his mouth, he thus has an incentive of at least several hundred thousand dollars not to damage that reputation. If he did already state that he would not break an oath to save the world, and you offered him a million dollars to go on record as saying that he would lie or break an oath for pragmatic reasons, I'd be surprised if he took you up on it. (But I'd be up for it, if you have the money...)
I don't believe that he's lying. And I don't believe that he's telling the truth. I believe Eliezer may be operating on a level of rationality where we actually need to regard him as a rational agent. And he's playing a game that he expects to go on for many iterations. That implies that, short of fMRI, knowing his intent is impossible.
It might be more rational for him to pretend to be less rational. Therefore, he possibly already has.
Prices or Bindings
Thanks! I had read that, but had forgotten about it. Perhaps EY's position makes more sense within timeless decision theory? Since it seems to be based on an absolute requirement for integrity of pre-commitment.
On the other hand, he did not express disapproval of sinking the ship to stop the German nuclear bomb. What if Haukelid had had to promise not to harm the ship, in order to get access to it?