Sounds promising! I'll hopefully have the time to put together a design/prototype of my own tomorrow.

I meant something like the difference between:

Either of those could work, but I'm worried that the steps that the latter option would require would easily make the player feel like she was doing tedious work that could easily have been automated instead. I'm not sure about that, though: getting to enter the data could also feel rewarding. We'll just have to experiment with it.

My biggest problem is that I have no idea how to actually score a player if he gets to come up with his own probabilities in a fictional world. Maybe the game needs to have some way of explicitly finding out the "right" values for some priors and correlations.

Well, if different beliefs have different consequences in the world ("if you believe the assassin is in the bell tower, go there to stop him") and the player is scored on his ability to achieve things in the world, that also implicitly scores him on probabilities that are maximally correct / useful. But this might not be explicit enough, if the player has no clue of what the probabilities should be like and feels like they're just hopelessly flailing around.