Sniffnoy comments on Desirable Dispositions and Rational Actions - Less Wrong

13 Post author: RichardChappell 17 August 2010 03:20AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (180)

You are viewing a single comment's thread. Show more comments above.

Comment author: Sniffnoy 18 August 2010 12:59:44AM 2 points [-]

For any decision theory, isn't there some hypothetical where Omega can say, "I've analyzed your decision theory, and I'm giving you proposition X, such that if you act the way your decision theory believes is optimal, you will lose?" The "Omega scans your brain and tortures you if you're too rational" would be an obvious example of this.

This isn't obvious. In particular, note that your "obvious example" violates the basic assumption all these attempts at a decision theory are using, that the payoff depends only on your choice and not how you arrived at it. Of course this is not necessarily a realistic assumption, but that is, IINM, the problem they're trying to solve.

Comment author: ocr-fork 18 August 2010 06:22:28AM 0 points [-]

This isn't obvious. In particular, note that your "obvious example" violates the basic assumption all these attempts at a decision theory are using, that the payoff depends only on your choice and not how you arrived at it.

Omega simulates you in a variety of scenarios. If you consistently make rational decisions he tortures you.

Comment author: Sniffnoy 18 August 2010 11:47:15PM *  1 point [-]

My reply to this was going to be essentially the same as my comment on bentarm's thread, so I'll just point you there.

Comment author: Psychohistorian 18 August 2010 01:33:21AM 0 points [-]

That does make it somewhat more useful, if that's the constraint under which it's operating. It still strikes me as probable that, insofar as decision theory A+ makes decisions that theory A- does not, there must be some way to reward A- and punish A+. I may well be wrong about this. The other flaws, namely the fact that actual decision makers do not encounter omniscient entities with entirely inscrutable motives that are unwaveringly honest, still seem to render the pursuit futile. It's decidedly less futile if Omega is constrained to outcome based reward/punishment.