Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

gwern comments on The Optimizer's Curse and How to Beat It - Less Wrong

44 Post author: lukeprog 16 September 2011 02:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (81)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 16 September 2011 02:06:22PM 14 points [-]

The central point of the optimizer's curse not one I have seen before and is a very interesting point.

Reminds me of the winner's curse in auctions - the selected bid is the one that is the highest and so most likely to be due to overconfidence/bias.

Comment author: malthrin 16 September 2011 03:21:19PM 4 points [-]

Yes, I recognized that similarity as well. As an aside, Fantasy Football (especially with an auction draft) is a great example to use when explaining these overestimation effects to laypeople.

Comment author: lessdazed 16 September 2011 04:40:16PM 5 points [-]

They were talking about the Lottery. Winston looked back when he had gone thirty metres. They were still arguing, with vivid passionate faces. The Lottery, with its weekly payout of enormous prizes, was the one public event to which the proles paid serious attention. It was probable that there were some millions of proles for whom the Lottery was the principal if not the only reason for remaining alive. It was their delight, their folly, their anodyne, their intellectual stimulant. Where the Lottery was concerned, even people who could barely read and write seemed capable of intricate calculations and staggering feats of memory. There was a whole tribe of men who made a living simply by selling systems, forecasts and lucky amulets.

--2001: A Space Odyssey (Homer, translated from ancient Latin)

Comment author: malthrin 16 September 2011 07:58:18PM *  5 points [-]

Interesting sourcing on that quote. I'm not sure what you meant to say with it, so I'll elaborate.

In fantasy sports, you begin by calculating an expected value for each player over the upcoming season. These values are used to construct your team in a draft, which is either turn-based (A picks a player, then B, then C) or auction-based (A, B, and C bid on players from a fixed initial pool of money). As the season goes on, you update your expected values with evidence from the past week's games in order to decide which players will be active and accrue points for your fantasy team.

The analogy should be obvious for most folks here. You're combining evidence to form a probability (how good was he last season? Is the new coach's game plan going to help or hurt his stats? Is he a particularly high injury risk?) and multiplying by utility to form a preference ranking. In an auction draft, the pricing mechanism even requires you to explicitly compute the expected utility values. When games are played, you update on evidence and revise your rankings.

Most people have a hard time relating to decision theory because it doesn't "feel like" what goes on in their head when they make decisions. Fantasy sports is a useful example because it makes the process explicit. I didn't fully realize how good a fit it is before this conversation - maybe I should write up an introductory rationality piece on this foundation.

Comment author: lessdazed 16 September 2011 08:51:55PM *  3 points [-]

The quote is from Orwell's 1984. The proles are generally ignorant, but good at tracking lottery numbers because it is a game. That's right, I just generalized from fictional evidence!

I figured if people are going to complain about the Burns quote, I'd give them something to really complain about. Wrong book with a date as a title, wrong author of an Odyssey, wrong language.

Fantasy sports is a great example of where this would be useful, and I can't think of a better analogy.