The issues you raised are interesting but actually make this a pretty good example of my problem - how do you account for weak evidence and assign it a proper likelihood. One way i am testing this is by taking an example which i think is agreed to be 'most likely' (that he existed as opposed to not existing). Then i want to work backwards and see if we there is a method for assessing probability that seems to work well on small scale questions, like probability's of minted coins and give me the expected answer when i add it all together.
At this point i am still trying to work out the objective priors issue. The method either needs to be immediately agreeable by all potential critics or have an open and fair way of arguing over how to formulate the answer. When i work that out i will move to the next stages although no guarantee i keep using the Alexander example.
My point was that 'probability of minted coins' isn't a much "smaller-scale" question than 'probability of Alexander', that is, it isn't much simpler and easier to decide.
In our model of the world, P(coins) doesn't serve as a a simple 'input' to P(Alexander). Rather, we use P(Alexander) to judge the meaning of the coins we find. This is true not only on the Bayesian level, where all links are bidirectional, but in our high-level conscious model of the world, where we can't assign meaning to a coin with the single word Alexander on it without already believing that Alexander did all the things we think he did.
There's very little you can say about these coins if you don't already believe in Alexander.
I posted before about an open source decision making web site I am working on called WikiLogic. The site has a 2 minute explanatory animation if you are interested. I wont repeat myself but the tl;dr is that it will follow the Wikipedia model of allowing everyone to collaborate on a giant connected database of arguments where previously established claims can be used as supporting evidence for new claims.
The raw deduction element of it works fine and would be great in a perfect world where such a thing as absolute truths existed, however in reality we normally have to deal with claims that are just the most probable. My program allows opposing claims to be connected and then evidence to be gathered for each. The evidence will create a probability of it being correct and which ever is highest, gets marked as best answer. Principles such as Occams Razor are applied automatically as long list of claims used as evidence will be less likely as each claim will have its own likelihood which will dilute its strength.
However, my only qualification in this area is my passion and I am hitting a wall with some basic questions. I am not sure if this is the correct place to get help with these. If not, please direct me somewhere else and I will remove the post.
The arbitrarily chosen example claim I am working with is whether “Alexander the Great existed”. This has the useful properties of 1: an expected outcome (that he existed - although, perhaps my problem is that this is not the case!) and 2: it relies heavily on probability as there is little solid evidence.
One popular claim is that coins were minted with his face on them. I want to use Bayes to find how likely a face appearing on a coin is for someone who existed. As I understand it, there should be 4 combinations:
The first issue is that there are infinite people who never existed and did not have a coin made. If I narrow it to historic figures who turned out not to exist and did not have a coin made it becomes possible but also becomes subjective as to whether someone actually thought they existed. For example, did people believe the Minotaur existed?
Perhaps I should choose another filter instead of historic figure, like humans that existed. But picking and choosing the category is again so subjective. Someone may also argue that woman inequality back then was so great that the data should only look at men, as a woman’s chance of being portrayed on a coin was skewed in a way that isn’t applicable to men.
I hope i have successfully communicated the problem i am grappling with and what i want to use it for. If not, please ask for clarifications. A friend in academia suggested that this touches on a problem with Bayes priors that has not been settled. If that is the case, is there any suggested resources for a novice with limited free time, to start to explore the issue? References to books or other online resources or even somewhere else I should be posting this kind of question would all be gratefully received. Not to mention a direct answer in the comments!