Wei_Dai comments on Common mistakes people make when thinking about decision theory - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (27)
The difference between Hofstadter and Eliezer is that Hofstadter couldn't make a convincing enough case for his assumptions, because he was talking about humans instead of AIs, and it's just not clear that human decision procedures are similar enough to each other for his assumptions to hold. Eliezer also thought his ideas applied to humans, but he had a backup argument to the effect "even if you don't think this applies to humans, at least it applies to AIs who know each others' source code, so it's still important to to work on" and that's what convinced me.
BTW, for historical interest, I found a 2002 post by Hal Finney that came pretty close to some of the ideas behind TDT:
I responded to Hal, and stated my agreement, but neither of us followed it up at the time. I even forgot about the post until I found it again yesterday, but I guess it must have influenced my thinking once Eliezer started talking about similar ideas.