I would say (mostly based on intuition, again) that we should assign a probability to our intuitions being correct vs. complicated, subtle arguments. We should take into account that our intuitions are often correct (depending on the issue in question) even if we can't always explain why. We should also take into account that on similar types of issues with complicated subtle arguments we might not be smart enough to determine how to resolve all the arguments (again, depending on the issue in question).
In some cases there will be so much evidence that our intuitions should bow, and in other cases there will be only weak arguments vs. powerful intuitions in which case (assuming it's a case where our intuitions have some weight) we'd probably say the intuitions should win.
What about a grey case where we don't know which one wins? In epistemic questions (which map better fits the territory), we can just leave it as unresolved, assigning probabilities close to 50% for both sides. In some decision questions, however, we must pick one side or the other, and in these cases my impression (I don't know too much decision theory) is that we essentially pick one side at random.
In the case of a pet religion / philosophy, we know that our intuitions are subject to a number of powerful biases which we need to take into account, and in many cases the arguments against our religion / philosophy might be very powerful. In the case of Pascal's Mugging decision problems, however, to the best of my knowledge there are only weak biases involved and very iffy arguments, so I'd think we should go with our intuitions.
One more point: Most of the arguments involving extreme cases of decision theory that I've seen (including Pascal's Mugging) start from the assumption that "this is obviously wrong, but why?". So we're anyway appealing to our intuition that it's wrong. Then we go on to say that "maybe it's wrong because of X", and we then extrapolate X back up the probability scale to less extreme cases and say that "well, if X is correct then we should counter-intuitively not worry about this case either". In which case we end up using an extrapolation of a tentative response to one intuition in order to argue against another intuition. I (intuitively?) think there's something wrong with that.
I agree with basically all of this.
It irritates me when people talk as though Pascal's Mugging or Wager is an obvious fallacy, but then give response which are fallacious themselves, like saying that the probability of the opposite is equal (it is not), or that there are alternative scenarios which are just as likely, and then give much less probable scenarios (e.g. that there is a god that rewards people for being atheist), or say that when you are dealing with infinities it does not matter which one is more probable (it does). You are quite correct that ...
Some people[1] are now using the term Pascal's mugging as a label for any scenario with a large associated payoff and a small or unstable probability estimate, a combination that can trigger the absurdity heuristic.
Consider the scenarios listed below: (a) Do these scenarios have something in common? (b) Are any of these scenarios cases of Pascal's mugging?
(1) Fundamental physical operations -- atomic movements, electron orbits, photon collisions, etc. -- could collectively deserve significant moral weight. The total number of atoms or particles is huge: even assigning a tiny fraction of human moral consideration to them or a tiny probability of them mattering morally will create a large expected moral value. [Source]
(2) Cooling something to a temperature close to absolute zero might be an existential risk. Given our ignorance we cannot rationally give zero probability to this possibility, and probably not even give it less than 1% (since that is about the natural lowest error rate of humans on anything). Anybody saying it is less likely than one in a million is likely very overconfident. [Source]
(3) GMOS might introduce “systemic risk” to the environment. The chance of ecocide, or the destruction of the environment and potentially humans, increases incrementally with each additional transgenic trait introduced into the environment. The downside risks are so hard to predict -- and so potentially bad -- that it is better to be safe than sorry. The benefits, no matter how great, do not merit even a tiny chance of an irreversible, catastrophic outcome. [Source]
(4) Each time you say abracadabra, 3^^^^3 simulations of humanity experience a positive singularity.
If you read up on any of the first three scenarios, by clicking on the provided links, you will notice that there are a bunch of arguments in support of these conjectures. And yet I feel that all three have something important in common with scenario four, which I would call a clear case of Pascal's mugging.
I offer three possibilities of what these and similar scenarios have in common:
In any case, I admit that it is possible that I just wanted to bring the first three scenarios to your attention. I stumbled upon each very recently and found them to be highly..."amusing".
[1] I am also guilty of doing this. But what exactly is wrong with using the term in that way? What's the highest probability for which the term is still applicable? Can you offer a better term?
[2] One would have to define what exactly counts as "direct empirical evidence". But I think that it is pretty intuitive that there exists a meaningful difference between the risk of an asteroid that has been spotted with telescopes and a risk that is solely supported by a priori arguments.