I'm late to this debate; it's been well analyzed on the meta level but I want to add something on the object level. The question of 'were coins of Alexander minted?', which you want as your input data, sounds like a very simple one, but it may not be. The analysis of ancient coins in large part assumes a known history and dating.

1. The coins called 'Alexander coins' today didn't carry the image of Alexander, but of Herakles. They had Alexander's name on them, and sometimes the title 'King', but that's just two words; not strong evidence of who exactly the coin-minters thought Alexander was and what he was supposed to have accomplished. So you have to separate 'some Greek King named Alexander existed' at one extreme, from 'he did what's commonly ascribed to him and is the person we think of as Alexander the Great' at the other.
2. These coins were minted in many countries for hundreds of years, becoming something of a common coinage. People probably kept minting them because they were a standard coin, not because they really wanted to honor Alexander 300 years after his death. So the existence of most coins isn't independent evidence.
3. When comparing coins minted of other people, you need to factor in the the different amounts of coins minted due to different technology and economy, the differential survival of the coins over time, and the probability of us finding them.
4. How do we identify a coin as being Alexandrian? Usually by matching it to other coins we've already identified. How do we identify the first Alexandrian coin we've ever seen? By noticing the name Alexander on it. What's the probability coins were minted of Alexander, perhaps bearing his likeness (and not that of Herakles), but without his name? Putting the name of a king on a coin (whether or not he was pictured) became common (but not universal) somewhat after Alexander.
5. There were five kings of ancient Macedonia named Alexander, the third of them being the Great. Not to mention some non-Macedonian Alexanders. Coins with the name Alexander don't specify which one is meant, or even 'of Macedonia'. (A minority was actually minted in Macedonia, but if lots of other countries were minting coins of a foreign monarch, who's to say Macedonia didn't as well?) So the dating of the coins has to be precise enough (to within 50 years or so), which can be really hard (by the time you've found a coin who knows where it's been?)

Clarification: I don't seriously doubt the historicity of Alexander, but I'm also not versed in the subject, so I'm just going by expert opinion. My point is more that coins are really complex and/or weak as a source of evidence, and handling P(Alexander existed|Coins of Alexander) is really hard.

I agree that we are not in agreement. And I do think that if we continue to respond to each other indefinitely, or until we agree, it will probably result in a fight. I admit that is not guaranteed, and there have been times when people that I disagree with changed their minds, and times when I did, and times when both of us did. But those cases have been in the minority.

"We are all trying to reach a certain goal and a truer map of reality helps us get there..." The problem is that people are interested in different goals and a truer map of reality is not always helpful, depending on the goal. For example, most of the people I know in real life accept false religious doctrines. One of their main goals is fitting in with the other people who accept those doctrines. Accepting a truer map of reality would not contribute to that goal, but would hinder it. I want the truth for its own sake, so I do not accept those doctrines. But they cannot agree with me, because they are interested in a different goal, and their goal would not be helped by the truth, but hindered.

Correct by whose definition? In a consistent reality that is possible to make sense of, one would expect evolved beings to start coming to the same conclusions.

I wouldn't necessarily expect that for the reasons given. You have given contrary opinion, not a counter argument.

From this question i assume you are getting at our inability to know things and the idea that what is correct for one, may not be for another. That is a big discussion but let me say that i premise this on the idea that a true skeptic realizes we can not know anything for sure and that is a great base to start building our knowledge of the world from.

I don't see how it addresses the circularity problem.

That vastly simplifies the world and allows us to build it up again from some very basic axioms. If

Or that. Is everyone going to be on the same axioms?

It is the case that your reality is fundamentally different from mine, we should learn this as we go. Remember that there is actually only one reality - that of the viewers.

The existence of a single reality isn't enough to guarantee convergence of beliefs for the reasons given.

Do you not see that you are assuming you will suddenly be able to solve the foundational problems that philosophers have been wrestling with for millennia.

There were many issues wrestled with for millennia that were suddenly solved. Why should this be any different?

That doesn't make sense. The fact that something was settled eventually doesn't mean that you probably problems are going to be settled at a time convenient for you.

I could ask me the opposite question of course but that attitude is not the one taken by any human who ever discover something worth while. Our chances of success may be tiny but they are better than zero, which is what they would be if no one tries. Ugh... i feel like i am writing inspirational greeting card quotes but the point still stands!

Yes I feel that you are talking in vague but positive generalities.

I'm pretty sure that you're actually asking some deep questions right now.

I'm not all that well-versed in epistemology or probability theory, but when you write:

A friend in academia suggested that this touches on a problem with Bayes priors that has not been settled.

I think this is a reference to the problem of priors.

I think 'a problem with Bayes priors that has not been settled' kind of understates the significance.

And:

The first issue is that there are infinite people who never existed and did not have a coin made. If I narrow it to historic figures who turned out not to exist and did not have a coin made it becomes possible but also becomes subjective as to whether someone actually thought they existed. For example, did people believe the Minotaur existed? Perhaps I should choose another filter instead of historic figure, like humans that existed. But picking and choosing the category is again so subjective. Someone may also argue that woman inequality back then was so great that the data should only look at men, as a woman’s chance of being portrayed on a coin was skewed in a way that isn’t applicable to men.

I believe this is referred to as the reference class problem. It seems that in a Bayesian framework, the reference class problem is something of a subproblem of the problem of priors. It seems that you're only trying to define a suitable reference class in the first place because you're trying to produce a reasonable prior.

It's my understanding that one approach to the problem of priors has been to come up with a 'universal' prior, a prior which is reasonable to adopt before making any observations of the world. One example is Solomonoff's algorithmic probability. It seems however than even this may not be a satisfactory solution, because this prior defies our intuition in some ways. For example, humans might find it intuitive to assign nonzero prior probability to uncomputable hypotheses (e.g. our physics involves hypercomputation), but algorithmic probability only assigns nonzero probability to computable hypotheses, an agent with this prior will never be able to have credence in uncomputable hypotheses. Another problem is that, with this prior, hypotheses are penalized for their complexity, but utility can grow far more quickly than complexity. Increasing the number of happy people in a program from 1000 people to 1,000,000 people seems to increase its utility a lot without increasing its complexity by much. Taking this up to larger and larger numbers that become difficult to intuitively comprehend, it may be that such a prior would result in agents whose decision-making is dominated by very improbable outcomes with very high utilities. We can also ask if this apparently absurd result is a point against Solomonoff induction or if it's a point against how humans think, but if we humans are thinking the right way, we still don't know what it is that's going right inside of our heads and how it compares to Solomonoff induction.

For any other readers, sorry if I'm mistaken on any of this, it is quite technical and I haven't really studied it. Do correct me if I've made a mistake.

Back to my point, I think that you accidentally hit upon a problem that doesn't seem to take too many prerequisites to initially run into, and that, after a bit of squinting, turns out to be way harder than it seems it should be at first glance, given the small number of prerequisites necessary to realize that the problem exists. Personally, I would consider the task of providing a complete answer to your questions an open research problem.

Stanford Encyclopedia of Philosophy's entry on Bayesian epistemology might be some good reading for this.

I don't recall Jaynes discussing it much. Anyone?

For him, I think the reference class is always the context of your problem. Use all information available.

A brief google for "jaynes reference class" turned up his paper on The Well Posed Problem.

http://bayes.wustl.edu/etj/articles/well.pdf

He analyzes the Bertrand Paradox, and finds that in the real world, the mathematical "paradox" is resolved by identifying the transformation group (and thereby prior) that in reality is applicable.

My take on this is that "non-informative priors" and "principle of indifference" are huge misnomers. Priors are assertions of information, of transformation groups or equivalence classes believed appropriate to the problem. If your prior is "gee I don't know and don't care", then you're just making shit up.

But picking and choosing the category is again so subjective.

No. Use all information available. What problem are you actually looking to analyze? What information do you have?

Someone may also argue that woman inequality back then was so great that the data should only look at men, as a woman’s chance of being portrayed on a coin was skewed in a way that isn’t applicable to men.

That may be some useful information to include. Willfully ignoring relevant information, or not seeing how to use some information that seems like it may be relevant does not mean that the problem is "subjective", it means that we are often lazy and confused. And that's fine.

Include what you can transform into meaningful probabilities.

That thinking is hard is not a problem unique to bayesian methods.

Sure, but why will they disagree?

From the correct perspective, it is more extraordinary that anyone agrees.

If I say there is 60% chance of x and you say no it is more like 70% then i can ask you why you think its 10% more likely. I know many will say "its just a feeling" but what gives that feeling? If you ask enough questions, i am confident one can drill down to the reasoning behind the feeling of discomfort at a given estimate.

Yes but that is not where the problems stop, it is where they get really bad. Object level disagreements can maybe be solved by people who agree on an epistemology. But people aren't in complete agreement about epistemology. And there is no agreed meta epistemology to solve epistemological disputes..that's done with same epistemology as before. And that circularity means we should expect people to inhabit isolated, self sufficient philosophical systems.

benefit of WL is it should help people get better at recognizing and understanding their subconscious feelings so they can be properly evaluated and corrected.

Corrected by whose definition of correct? Do you not see that you are assuming you will suddenly be able to solve the foundational problems that philosophers have been wrestling with for millennia.

,If you do not agree, it would be really interesting to hear your thoughts on this. Thanks

I don't agree. If you're right, we can do it right here and now, since we do not agree, which means that we are giving different probabilities of your project working -- in particular, I say the probability of your project being successful is very close to zero. You presumably think it has some reasonable probability.

I think the probability is close to zero because trying to "drill down" to force agreement between people results in fights, not in agreement. But to address your argument directly, each person will end up saying "it is just a feeling" or some equivalent, in other words they will each support their own position by reasons which are effective for them but not for the other person. You could argue that in this case they should each adopt a mean value for the probability, or something like that, but neither will do so.

And since I have given my answer, why do you think there is a reasonable probability that your project will succeed?

I think reading Tedlocks book about Superforcasting could help you because it talks about how his Superforcasters made their decisions and outperformed the CIA analysts.

References to books or other online resources or even somewhere else I should be posting this kind of question would all be gratefully received.

How about http://stats.stackexchange.com/?

With the Alexander the Great issue, you not only have the question of the probability that someone would have a coin minted and so on, but also the probability that he actually did have a coin minted with his face on it, given that someone says he did. And you cannot assign that to 100% even if you have seen the coin yourself, because it may not have been intended to be the face of Alexander the Great, but someone else.

And basically all of those probabilities are subjective, and cannot be made objective. So your project might work for an individual to map out their own beliefs, allowing them to assign subjective probabilities to things they consider priors. But it cannot work for a community, because people will disagree with the priors you assign, and so there will be no reason for them to agree with the probabilities your program comes up with.