Seems interesting, but almost everything of it flew over my head. Still, even only the table of the biases might be of use in self-critique; like, am I unpacking the hypothesis A into several unlikely smaller hypotheses (A1, A2...) and so underestimating the probability of A because I see it as a conjunction of events of small probability?.. am I, on the other hand, thinking of A as a disjunction, and so overestimating its probability?..
It is hard (at least, for me) to even judge whether I consider something a disjunction or a conjunction - I usually have my 'hypotheses' as chunks of thought, of variable length, just running their course from the spark of noticing (something observed or imagined) to the end (often a variation of 'who cares').
(And if shower-thoughts are so helpful in thinking up alternative explanations and 'just random stuff'... then I hate having had long hair for most of my life. It meant head colds in the cold season after every second bath! Hooray for short, easily driable hair!
OTOH, when we had to digest, in college, chunks that were noticeably too large - like, cramming for a test on Arthropoda, or for the final test on virology, or even a seminar on photosynthesis, - sometime half-way to the end I began having this feeling, 'stop, this is too elaborate, rewind, this isn't beautiful anymore, quit'. And whoa, it turned out that Arthropoda were even more "monstrous" than I could take in (= even more variable and specialized), viruses were "Just Doing It" with no regard for the "everyday" ranges of any nameable constraints, and the lovely, elegant photosynthesis seemed a total mess when coupled to the rest of the plant.
I mean, first, they give you something that obviously wouldn't work as a human plot, then they show you that yes, it works, and then they tell you 'oh btw, you graduated now, go do your thinking for yourself, there's a good chap, and don't forget to penalize complex hypotheses! Because conjunctions!' Life is unfaaaair.)
I mean, thanks for the link, it's pretty scary:)
Which is to say, I think there is a big problem in teaching people (biologists, at least) to think up hypotheses. In that, the reasoner knows the final product must be of some non-trivial complexity, in the colloquial sense of the word, but that he isn't really allowed to put forward anything too complicated, because it's too easy to imagine a just-so story. (So now I will imagine myself a just-so story, because duh.)
One way to tinker with research is to think of your tools, whether material or mathematical, as part of the picture instead of the brush and ...