This does raise a question about why no such experiment has been attempted. If one really thinks that one's rationality skills are that much better and that there's that much low hanging fruit then this is an obvious thing to try.
My gut feel is that Bayesianism is one of the many useful tools in research, but it is no substitute for education, experience, creativity, patience or tenacity. I am not at all certain that it helps one identify the proverbial low-hanging fruit in any meaningful way. I would be very happy if komponisto or anyone else proved me wrong (experimentally, not argumentatively).
Today's post, Class Project was originally published on 31 May 2008. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Einstein's Superpowers, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.