JonathanLivengood comments on Some potential dangers of rationality training - Less Wrong

18 Post author: lukeprog 21 January 2012 04:50AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (48)

You are viewing a single comment's thread. Show more comments above.

Comment author: JonathanLivengood 23 January 2012 04:07:00AM 0 points [-]

If you look at the context where your strategy is winning, it looks rational, so this example does not apply.

I think maybe we're talking past each other, then. I thought the idea was to imagine cases where the algorithm or collection of behaviors generated by the algorithm is rational even though it has sub-parts that do not look rational. You are absolutely right when you say that in-context, the play on Game B is rational. But that's the whole point I was making. It is possible to have games where optimal play globally requires sub-optimal play locally.

That is why I put "irrational" in those scare quotes in my first comment. If a behavior really is optimal, then any appearance of irrationality that it has must come from a failure to see the right context.