Thank you for the clarification. I'm still confused about something, and to explain where I was getting stuck, I think it may have been the deciders prediction of the Muggers/expected fact of the universes response to the question "Can you show me more evidence that you are a Matrix Lord(or a fact of the universe set up with comparable probability and utility.)"
For instance, if the Mugger might say:
A: "Sure, let me open a firey portal in the sky."
B: "Let me call the next several coinflips you toss."
C: "No, you'll just have to judge based on the current evidence."
D: "You were correct to question me, this was actually a scam."
E: "You question me? The offer is now invalidated and/or I have killed those people."
F: "You can't investigate this right now because your evidence gathering abilities are too low, but you could use these techniques to increase your maximum evidence gathering abilities. With sufficient repeated application, you would be able to investigate the original problem."
On the other hand, a fact of the universe may be such that:
A: Further investigation leads to sudden dramatic shifts such as firey portals.
B: Further investigation leads to more evidence that it's right, but nothing dramatic.
C: Further investigation leads nowhere new. You'll have to decide on current evidence.
D: Further investigation shows worrying about this was a waste of time.
E: Further investigation caused you to lose the opportunity: it was time sensitive.
F: Further investigation leads you to better investigative techniques, but you still can't actually investigate the original problem. Perhaps you should try again?
And I was thinking "If it's impersonal and simple, such as the box, maybe you may be stuck with C. But Foolish, Sadistic, or Testing Lords may give you anywhere from A-F." (A testing lord in particular seems likely to give you scenario F.)
However, from your, Stuart_Armstrong and ArisKatsaris's replies, this is not actually the area that is currently of concern, but I'm still somewhat confused about which position A-F I should be taking, whether it is just irrelevant to the problem and all would be handled the same, or whether some/each represents an entirely separate scenario which should be handled on it's own.
Maybe material for a further post...
There are two separate reasons to reject Pascal's mugger's demands. The first one is if you have a system of priors or a method of updating that precluded you from going along with the deal. The second reason is that if it becomes known that you accept Pascal's mugger situations, people are going to seek you out and take advantage of you.
I think it's useful to keep the two reasons very separate. If Pascal's mugger was a force of nature - a new theory of physics, maybe - then the case for keeping to expected utility maximisation may be quite strong. But when there are opponents, everything gets much more complicated - which is why game theory has thousands of published research papers, while expected utility maximisation is taught in passing in other subjects.
But does this really affect the argument? It means that someone approaching you with a Pascal's mugging today is much less likely to be honest (and much more likely to have simply read about it on Less Wrong). But that's a relatively small shift in probability, in an area where the number are already so huge/tiny.
Nevertheless, it seems that "reject Pascal's muggings (and other easily exploitable gambles)" may be a reasonable position to take, even if you agreed with the expected utility calculation. First, of course, you would gain that you reject all the human attempts to exploit you. But there's another dynamic: the "Lords of the Matrix" are players too. They propose certain deals to you for certain reasons, and fail to propose them to you for other reasons. We can model three kinds of lords:
Precommitting to rejecting the mugging burns you only with the foolish lords. The sadistic lords won't offer an acceptable deal anyway, and the testing lords will offer you a better deal if you've made such a precommitment. So the gain is the loss with (some of) the foolish lords versus a gain with the testing lords. Depending on your probability distribution over the lord types, this can be a reasonable thing to do, even if you would accept the impersonal version of the mugging.