There are domains where it's easy to perform experiments (physics, chemistry) and others where it's unfeasible (biology, economy) or impossible (psychology).
The quality of scientific understanding in these different domains is necessarily different. Has there been any thoughts or study devoted to the subject of doing statistics or Bayesian learning where you can suffer from lack of feedback or hysteresis? Is there a mathematics for doing science in low feedback domains?
The whole point of Bayesianism is that you get as much information as possible from a small amount of data. It works perfectly well in noisy domains.
The recent post on Knightian Uncertainty may or may not be relevant to your interests - it's not the same thing but it seems like it might be related to what you were really getting at.
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.