Yvain comments on 3 Levels of Rationality Verification - Less Wrong

43 Post author: Eliezer_Yudkowsky 15 March 2009 05:19PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (182)

You are viewing a single comment's thread.

Comment author: Yvain 15 March 2009 07:42:07PM 2 points [-]

Here is a stupid one: Detective stories. Like Encyclopedia Brown, but subtler. And with false leads. I don't think normal mass-market detective stories would work, because they may try to deliberately choose an irrational answer to surprise you. But special ones written by rationalists for rationalists could be a fun distraction if nothing else.

Comment author: rwallace 15 March 2009 08:25:49PM 3 points [-]

That still has the problem that it doesn't test for lack of bias, but for having bias that matches that of the people who wrote the stories. I suggest instead using real cases - and not taken from the media, because that means selection bias, but taking all the cases from the files of a particular police department during a particular span of time.

Comment author: Eliezer_Yudkowsky 15 March 2009 08:27:11PM 1 point [-]

What true stories can we use besides police cases? (Also, note that in this case you're only testing for being as smart as the police or making the same judgments as the jury - even using cases with a confession may get you false confessions.)

Comment author: CarlShulman 15 March 2009 09:10:58PM 5 points [-]

You can take cases with enough evidence to overdetermine the result, and then subtract pieces.

Comment author: rwallace 15 March 2009 08:39:27PM *  2 points [-]

Point. Still, we've been recording lots of different kinds of events for a long time. Off the top of my head, other kinds of historical data that could be useful here:

Medical cases, minor scientific controversies, engineering projects, battles, the stock market, markets in general, expeditions.