It occurs to me that I may have made my answer to your post a little long,but I figured that when I was unsure of how to judge something, more tests are better than fewer tests.
Starting with the basic tests from Wikipedia:
1: Does the model have organelles?
2: Does the model have cytoplasm?
3: Are the models organelles and cytoplasm enclosed by a cell membrane?
4: Does the model obtain its food through phagocytosis?
5: Is it a Heterotroph?
6: Does it have a single large tubular pseudopod at the anterior end, and several secondary ones branching to the sides?
It would probably be reasonable for a fake amoeba simulation to have all of these.
Then you start applying more tricky bits, like:
7: "Amoeba's are supposed to be able to recover from a forcible division if the nucleus is intact. If I remove some of the cytoplasm and keep the nucleus intact, does the simulation recover?"
8: "Cytoplasm is supposed to contain water. Does the bit of cytoplasm I extracted contain water?"
9: "Does the water divide into Hydrogen and Oxygen via electrolysis?"
Also, be sure to test things where the answer is expected to be no, as well, in case the simulation is just programmed to spit out "Yes." repeatedly.
10: "Does the Amoeba contain multiple nuclei?"
11: "Does the Amoeba have a human nervous system?"
12: "Is the Amoeba more than 99% Salt?"
Then ask 3 more questions which are LIKE question 7-12, but which I am not writing down, in case the person simply Googled for "Questions people have ever asked about Amoebas." and recorded appropriate answers.
Now make sure to ask questions which aren't Yes/No questions, because Yes/No questions don't actually accumulate bits of evidence all that quickly.
For instance
16: "How many Base pairs does the Ameoba have in it's genome?"
17: "How long is the Amoeba while moving?"
18: "How long is the Amoeba while not moving?"
19: "How fast can the Amoeba move?"
20: "Amoebas contain water. Water can put out forest fires. How many of your simulated Amoebas would I have to drop from 100 meters above a forest fire currently covering 100 acres of burning forest surrounded by 1000 acres of unburnt forest to put out the fire before the unburnt forest burns up?" (This seems like a bizarre question, but I am trying to be thorough. Also, someone's reaction to a question that they would never be expected to have an answer to under normal circumstances can itself be revealing.)
And again, ask a few more unlisted questions here, as well, including at least one other question just as odd as the forest fire question. (again, they might have prepared rote answers to all known Ameoba related items.) Also, you may want to ask some other later questions first if you suspect you might have a limited number of questions.
Now it is possible that the person will say "I'm only simulating an Amoeba, there isn't a simulated forest that I can run forest fire tests on." and "I'm only simulating an Amoeba, there isn't anything in the simulation which can be used to generate electricity to do electrolysis on water." and "I'm only simulating an Amoeba, there isn't anything in the simulation which can be used to forcibly divide it." And you would expect SOME of these. For instance, I would be surprised if anyone who was simulating an Amoeba ALSO simulated a forest for forest fire tests using the water inside that Amoeba. That might for instance indicate that they DID have unlimited computing power and simply simulated everything. Or that they were faking answers. It depends on the amount of time the answer takes and the type.
But you could say that to anything. "I'm only simulating an Amoeba, there isn't any light in the simulation to see it." "I'm only simulating an Amoeba, there are no objects in the simulation to test it's permeability to objects." "I'm only simulating an Amoeba, there is nothing in the simulation which can be used to judge how fast the Amoeba is."
If the mugger seems to be saying that or something similar to EVERYTHING, it's very probably a fake, particularly if it sounds almost like a rote answer with no delays.
Given an answer to all of those questions, and a list of objections if any, and timing knowledge about the questions, I think I would have a fairly good grasp of whether the model simulating an Amoeba was valid (or simulating anything really, I can change the specific questions given something else.) given about 25 questions or so. Of course there are edge cases. It is entirely possible to come up with a list of answers which I will see, and then come to the wrong conclusion about. but it should take enough effort that it would be easier to get the mugged amount of money from someone else rather than making a series of incredibly elaborate ruses just for a few dollars.
Thank you, you have illustrated my point that the test to even consider the mugger seriously, let alone take his claims at face value is easy to devise, but hard to pass, unless one is indeed an expert in simulating living things very convincingly.
It would seem rational to accept any argument that is not fallacious; but this leads to consideration of problems such as Pascal's mugging and other exploits.
I've had a realization of a subconscious triviality: for me to accept an argument as true, it is not enough that I find no error in it. The argument must also be so structured that I would expect to have found an error if it was invalid (or I myself must make such structured version first). That's how mathematical proofs work - they are so structured that finding an error requires little computational power (only knowledge of rules and reliability); in the extreme case an entirely unintelligent machine can check a proof.
In light of this I propose that those who want to make a persuasive argument should try to structure the argument so it'd be easy to find flaws in it. This also goes for the thought experiments and hypothetical situations. Those seem rather often to be constructed with entirely opposite goal in mind - to obstruct the verification process or to try to prevent the reader from trying to find flaws.
Something else tangentially related to the arguments. The faulty models are the prime cause of decision errors; yet the faulty models are the staple of thought experiment; nobody raises an eyebrow as all models are ultimately imperfect.
However, to accept an argument based on imperfect model one must be capable of correctly propagating the error and estimating the error in the final conclusion, as a faulty model may be so constructed as to itself differ non substantially from the reality but in such a way that the difference diverges massively along the chain of reasoning. My example of this is the Trolley Problems. The faults of original model are nothing out of ordinary; simplified assumptions of the real world, perfect information, etc. Normally you can have those faults in model and still arrive at reasonably close outcome. The end result is throwing of fat people onto tracks, cutting up of travellers for organs, and similar behaviours which we intuitively know we could live a fair lot better without. How that happens? In real world the strongly asymmetrical relations of form 'death of 1 person saves 10 people' are very rare (as an emergent property of complexity of the real world that is lacking in the imaginary worlds of trolley problems), while the decision errors are not nearly so rare, so most of people killed to save others would end up killed in vain.
I don't know how models can be structured as to facilitate propagation of model's error. But it seems to be necessary for arguments based on models to be convincing.