This is a linkpost for http://gershmanlab.webfactional.com/pubs/Dasgupta17.pdf
While only having read the abstract at the moment, this seems to confirm my belief that one should generate a large amount of hypotheses when one wants a more rigorous answer to a question. I've started doing this in my PhD research, mostly by compiling others' hypotheses, but also by generating my own. I've been struck by how few researchers actually do this. However, the researchers who indeed do consider multiple hypotheses (e.g., in my field one major researcher who does is Rolf Reitz) earn greater respect from me.
Also, hypothesis generation is definitely non-trivial in real scientific domains. Both generating entirely new hypotheses and steelmanning existing hypotheses are non-trivial. It doesn't matter if your scientific method will converge to the right hypothesis if it's in your considered set if most sets don't contain the "correct" hypothesis...
Very interesting paper. I will be reading this closely. Thanks for posting this link.
This is a real elephant in the room. It's been mentioned a few times here, but it remains a major epidiment to Bayes is the Only Epistemology you need, and other cherished notions.
- The problem of ignored hypotheses with known relations
The biggest problem is with the “hypotheses and logical relations” setup.
...The setup is deceptively easy to use in toy problems where you can actually list all of the possible hypotheses. The classic example is a single roll of a fair six-sided die. There is a finite list of distinct hypotheses one could have about t