timtyler comments on Comments for "Rationality" - Less Wrong

2 Post author: Eliezer_Yudkowsky 16 March 2009 10:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (38)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 18 March 2009 11:09:27AM 5 points [-]

Are you not getting the point? Agents can correctly apply inductive and deductive reasoning, but draw the wrong conclusion - because of their priors, or because of misleading sensory data. Rationality is about reasoning correctly. It is possible to reason correctly and yet still do badly - for example if a hostile agent has manipulated your sense data without giving you a clue about what has happened. Maybe you could have done better by behaving "irrationally". However, if you had no way of knowing that, the behaviour that led to the poor outcome could still be rational.

Comment author: Vladimir_Nesov 18 March 2009 01:17:32PM *  1 point [-]

I absolutely agree with this point. Rationality in this sense is that truth-engine I named in the comment you replied to: it's built for a range of possible environments, but can fail in case of an unfortunate happenstance. As opposed to having an insane maintainer who is convinced that the engine works when in fact it doesn't, not just on the actual test runs, but on the range of possible environments for which it's supposedly built. When you are 90% sure that something will happen, you expect it NOT to happen 1 time in 10.