"The coin's bias is not a random variable! It's a fixed fact! If you repeat the experiment, it won't come out to a 0.5 long-run frequency of heads!"
You're repeating the wrong experiment.
The correct experiment for a frequentist to repeat is one where a coin is chosen from a pool of biased coins, and tossed once. By repeating that experiment, you learn something about the average bias in the pool of coins. For a symmetrically biased pool, the frequency of heads would approach 0.5.
So your original premise is wrong. A frequentist approach requires a series of trials of the correct experiment. Neither the frequentist nor the Bayesian can rationally evaluate unknown probabilities. A better way to say that might be, "In my view, it's okay for both frequentists and Bayesians to say "I don't know.""
Eliezer:
You're repeating the wrong experiment.
The correct experiment for a frequentist to repeat is one where a coin is chosen from a pool of biased coins, and tossed once. By repeating that experiment, you learn something about the average bias in the pool of coins. For a symmetrically biased pool, the frequency of heads would approach 0.5.
So your original premise is wrong. A frequentist approach requires a series of trials of the correct experiment. Neither the frequentist nor the Bayesian can rationally evaluate unknown probabilities. A better way to say that might be, "In my view, it's okay for both frequentists and Bayesians to say "I don't know.""