I think the key point here is that there is uncertainty in both the distribution of coins and the outcome of any particular coin flip. The Bayesian answer is that you roll these two sources of uncertainty into a single uncertainty over the outcome of the next coin flip.
If this were a real world situation then your evidence would (ideally) include such unwieldy things as facts about your possible motivations, all the real-world facts about the physics of coin flips, and so on. Bayesianism tells us that there is a unique answer in the form of a probability for the next coin to be heads, but obviously the math to figure that out is enormously complicated.
If you don't want to deal with all this then you can pick some nicer mathematical starting point, like that the you have a uniform distribution over how biased the coins are, or a beta distribution, or some such. In this case do as follows
Let h be some hypothesis about what distribution the unfair coins follow
Write out P(h) according to the assumptions you made. A reasonable choice is the maximum entropy distribution (which in this case is uniform over the parameter p)
Let D be the data that you observed (100 heads). Write out P(D | h) - this will be a straightforward binomial distribution in this problem.
Write out P(h | D) - give a hurrah for Bayes rule at this point
Let T be the event of getting tails on the next flip. Write P(T | D) by marginalising over the hypotheses like integral-over-h P(T, h | D) = integral-over-h P(T | h) P(h | D)
That's it!
Bayesianism tells us that there is a unique answer in the form of a probability for the next coin to be heads
I'm obviously new to this whole thing, but is this a largely undebated, widely accepted view on probabilities? That there are NO situations in which you can't meaningfully state a probability?
For example, let's say we have observed 100 samples of a real-valued random variable. We can use the maximum entropy principle, and thus use the normal distribution (whcih is maximal-entropy for unbounded reals). We then use standard methods to estimate popu...
Suppose I tell you I have an infinite supply of unfair coins. I pick one randomly and flip it, recording the result. I've done this a total of 100 times and they all came out heads. I will pay you $1000 if the next throw is heads, and $10 if it's tails. Each unfair coin is entirely normal, whose "heads" follow a binomial distribution with an unknown p. This is all you know. How much would you pay to enter this game?
I suppose another way to phrase this question is "what is your best estimate of your expected winnings?", or, more generally, "how do you choose the maximum price you'll pay to play this game?"
Observe that the only fact you know about the distribution from which I'm drawing my coins is those 100 outcomes. Importantly, you don't know the distribution of each coin's p in my supply of unfair coins. Can you reasonably assume a specific distribution to make your calculation, and claim that it results in a better best estimate than any other distribution?
Most importantly, can one actually produce a "theoretically sound" expectation here? I.e. one that is calibrated so that if you pay your expected winnings every time and we perform this experiment lots of times then your average winnings will be zero - assuming I'm using the same source of unfair coins each time.
I suspect that the best one can do here is produce a range of values with confidence intervals. So you're 80% confident that the price you should pay to break even in the repeated game is between A80 and B80, 95% confident it's between A95 and B95, etc.
If this is really the best obtainable result, then what is a bayesianist to do with such a result to make their decision? Do you pick a price randomly from a specially crafted distribution, which is 95% likely to produce a value between A95..B95, etc? Or is there a more "bayesian" way?