I tried to solve it on my own, but haven't been able to so far. I haven't been able to figure out what sort of function someone who knows that I'm using UDT will use to predict my actions, and how my own decisions affect that. If someone knows that I'm using UDT, and I think that they think that I will cooperate with anyone who knows I'm using UDT, then I should break my word. But if they know that...
In general, I'm rather suspicious of the "trust yourself" argument. The Lake Wobegon effect would seem to indicate that humans don't do it well.
If you're so smart, why ain't you a rock? :P
And yeah, at some level you have to be checking for whether or not you are proving what the UDT agent will do - if you prove it you're safe, and if you don't you're not. The trouble is that checking for the proof can contain all the steps of the proof, in which case you might get things wrong because your search wasn't checking itself! So one way is to check for the proof in a way that doesn't correlate with the specific proof. "Did I check any proofs? No? Better not trust anyone."
Today's post, Prices or Bindings? was originally published on 21 October 2008. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Ethical Injunctions, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.