LESSWRONG
LW

Comment Permalink

To me singularity and cryonics are two different beasts.

The singularity is too far into the fog of the future to be taken seriously today. Maybe when there are some significant advances in the relevant fields, whenever that might happen. So far, even a single neuron is not fully simulated. The progress might go so many different ways, I find the singularity being just one of the many possible directions, and I know that my imagination is not up to par to even consider most of them.

On the other hand, cryonics is basically a bet that a frozen brain can potentially be fully restored. We already know that single cells can be (sperm banks do it on an industrial scale), and there is some success with organs and even organisms. So we might be only a few steps away from being able to repair the damage done by freezing.

adamisom13y30

And that's why FAI is such a big deal: precisely because it's "far into the fog of future" in the sense that we don't have any control on the outcome (unless we align its values very precisely... (read more)

See in context

32 On accepting an argument if you have limited computational power.

by Dmytry

11th Jan 2012

2 min read

32

It would seem rational to accept any argument that is not fallacious; but this leads to consideration of problems such as Pascal's mugging and other exploits.

I've had a realization of a subconscious triviality: for me to accept an argument as true, it is not enough that I find no error in it. The argument must also be so structured that I would expect to have found an error if it was invalid (or I myself must make such structured version first). That's how mathematical proofs work - they are so structured that finding an error requires little computational power (only knowledge of rules and reliability); in the extreme case an entirely unintelligent machine can check a proof.

In light of this I propose that those who want to make a persuasive argument should try to structure the argument so it'd be easy to find flaws in it. This also goes for the thought experiments and hypothetical situations. Those seem rather often to be constructed with entirely opposite goal in mind - to obstruct the verification process or to try to prevent the reader from trying to find flaws.

Something else tangentially related to the arguments. The faulty models are the prime cause of decision errors; yet the faulty models are the staple of thought experiment; nobody raises an eyebrow as all models are ultimately imperfect.

However, to accept an argument based on imperfect model one must be capable of correctly propagating the error and estimating the error in the final conclusion, as a faulty model may be so constructed as to itself differ non substantially from the reality but in such a way that the difference diverges massively along the chain of reasoning. My example of this is the Trolley Problems. The faults of original model are nothing out of ordinary; simplified assumptions of the real world, perfect information, etc. Normally you can have those faults in model and still arrive at reasonably close outcome. The end result is throwing of fat people onto tracks, cutting up of travellers for organs, and similar behaviours which we intuitively know we could live a fair lot better without. How that happens? In real world the strongly asymmetrical relations of form 'death of 1 person saves 10 people' are very rare (as an emergent property of complexity of the real world that is lacking in the imaginary worlds of trolley problems), while the decision errors are not nearly so rare, so most of people killed to save others would end up killed in vain.

I don't know how models can be structured as to facilitate propagation of model's error. But it seems to be necessary for arguments based on models to be convincing.

Bounded Rationality

Personal Blog

32

New Comment

85 comments, sorted by

top scoring

Click to highlight new comments since: Today at 5:36 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]malthrin13y300

There's a similar guideline in the software world:

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.

[-]cousin_it13y120

Your first point seems interesting. How specifically should we go about structuring arguments to make flaws easy to find?

8fiddlemath13y

Some ideas: The logical structure of an argument could be more clearly demonstrated by argument diagramming techniques. Wherever possible, seek to replace intuition with explicit evidence. Similarly, try to track the definitions of key concepts, ensuring that every term is fleshed out in more concrete terms, somewhere. Thus, issues of usage are replaced by pointers to specifics. None of these are new ideas on LW, but they should ask help this goal.

5Dmytry13y

Well, one important thing to avoid is the type of argument that requires massive exhaustive search through a vast space to find a flaw. It can be perhaps escaped by declaring such arguments void unless the full exhaustive search has been performed by a computer. I had some thoughts for the models. An argument based on the imperfect model of real world should outline imperfections of the model and then propagate the imperfections along with the chain of reasoning, as to provide upper bound on the final error. It is often done with computer simulations of e.g. atmosphere, where you can e.g. run simulation a lot of times with different values that are within the error range and look at the spread of the outcome.

3TimS13y

Use precise language, with explicit and/or clear definitions? I admit that "Write better" is not often helpful advice, but it is good advice.

[-]kilobug13y100

To me the solution to this problem is to not rely too much on raw consequentialism for dealing with real-life situation. Because I know my model of the world is in perfect, that I lack computing power to track all the consequences of an action and evaluate their utility, because I don't even know my own utility function precisely.

So I'm trying to devise ethical rules that come partly from consequentialism, but also taking into consideration lessons learned from history, both my own personal experience and humanity's history. And those rules for example say I should not kill someone, even if I think it'll save 10 lives, because usually when you do that, either you kill the person and fail to save the 10 others, or you failed to think to a way to save the 10 without killing one, or you create far-reaching consequences that'll at the end cost more than the 10 saved lives (for example, breaking the "don't kill" taboo, and leading for people to follow your example even in cases when they'll fail to save the 10 persons). That's less optimal than using consequentialism wisely - but also much less error-prone, at least to me, than trying to wield a dangerous tool that I'm not sm... (read more)

1vi21maobk9vp13y

More precisely, the core of our current best available (but still known to be flawed) physics are QM and GR and we do not even have a consistent model fully incorporating both.

1Eugine_Nier13y

Furthermore, we can't model anything more complicated then a hydrogen atom with QM without resorting to approximations, and by the time you get to something as complicated as bulk matter or atomic nuclei of heavy elements, we can't even verify that the predictions of QM are what we in fact observe.

0vi21maobk9vp13y

Very true, but we can test at least some multiple-particle predictions by attempting to build a small quantum computer

0DanielLC13y

From what I understand, we have more than one. We just don't know which, if any, is correct.

0vi21maobk9vp13y

We have some plans (including a few radically different from everything we are used to - which is good) how to build a model. I wouldn't call these plans models of anything yet, because QM and GR can help us predict the behaviour of precise tools we use, and these plans are not yet concrete enough to allow useful modelling.

0A1987dM13y

And some of them have so damn many free parameters that it would be hard to rule them out but they have hardly any predictive power.

0DanielLC13y

I think the biggest problem with killing someone is that you're likely to get arrested, which prevents you from saving hundreds of lives. In general, the best way to get something done is to pay someone who's better at it than you. As such, you can fairly accurately simplify it into thinking about how to earn money, and thinking about where to donate that money. These are generally things you can think about once, rather than think about on a case-by-case basis.

0kilobug13y

That specialization of labor does a lot of help doesn't mean that extreme specialization still does a lot of help. There are so many issues involved with letting someone else do something for you (finding/chosing the person, trust, explaining what you have to do, moving the person to the place, schedule/calendar issues, negotiating the price, legal issues, ...) that for many things, it's less efficient to pay someone to do it than to do it yourself, even if for the core of the task, a specialized person would be more efficient. Also, you've to consider willpower/akrasia/enjoyment related issues. For example, many people will feel much more motivated when fixing your own house than when fixing someone else house, so even if you need more time to do it than a professional, you could still fill better doing it yourself, than working (even less) extra hours in your job and paying someone to do it. Oversimplifying things like that just doesn't work in real life. And finally, you've to consider emergency situations. Trolley like situations are emergency situations, like if you see someone being mugged, or someone drowning of whatever, in those, you just don't have the option to pay someone to act for you.

0DanielLC13y

There would be vastly more things like this if specialization wasn't normal. That's what I meant when I said that it works better the more it's done. There are things it's better to do yourself, but most things aren't like that. The benefits from a situation not covered by the rule I gave earlier are very small. If you're in a situation where acting would be remotely dangerous, don't. If acting would be perfectly safe, go ahead. If it would be very slightly dangerous, then you're likely to be better off doing a Fermi calculation.

0thomblake13y

I'm not sure this generalizes well - would this work if everybody was doing it? (It might).

0DanielLC13y

Without specialization of labor, the world simply would not support this many people. Billions would die. It generalizes well. In fact, the more people do it, the better it works.

0thomblake13y

But I wonder whether thinking "How can I earn money?" gets specialization of labor as well / better than thinking "What's interesting to me?" It might result in people trying and failing to do things that pay a lot, rather than try and succeed at things they're well-suited for.

0DanielLC13y

If you know that will be a problem, I think you're smart enough to figure out that you have to do something you're interested in. If not, you're not going to come up with this as a guideline in the first place.

[-]Oscar_Cunningham13y60

The faulty models are the prime cause of decision errors; yet the faulty models are the staple of thought experiment...

The purpose of thought experiment is to analyse our theories in extreme situations. The understanding this gives can then be useful in non-extreme situations. An analogy to mathematics: When graphing y=x/(x^2+1) it is useful to consider the value of y as x goes to infinity, even if we only need a sketch for -2<x<2. Trolley Problems allow us to focus on the conflict between intuition and utilitarianism. The understanding thought ex... (read more)

4thomblake13y

Yes, though it is a standard response that weird situations are weird, and we should not expect our tools for dealing with everyday situations to apply. (cf. the ethical literature on "desert island cases")

4Oscar_Cunningham13y

I'd respond to that thusly: In order to extend our tools a little bit (as is necessary to deal with new situations), it helps to imagine extending them further than necessary. Otherwise multiple extensions in series will render them hopeless.

3Dmytry13y

Well, we don't blame special relativity for seeming to fail with all the thought experiments involving objects moving faster than speed of light. edit: that is to say, it is important that thought experiments remain within certain bounds. In the case of the trolley problems, the small difference between the assumptions and real world (neglecting the small false positive rate while focussing on the extremely low probability scenario) turn out to lead to massively incorrect result which is then counter intuitive. edit: and indeed, there are things which are correct but counter intuitive. However most of the things which are counter intuitive are also wrong; Earth being a torus is very counter intuitive and wrong, ditto for the saddle shape, etc.

0CG_Morton13y

That's hardly a critique of the trolley problem. Special relativity itself stipulates that it doesn't apply to faster-than-light movement, but a moral theory can't say "certain unlikely or confusing situations don't count". The whole point of a moral theory is to answer those cases where intuition is insufficient, the extremes you talk about. Imagine where we'd be if people just accepted Newtonian physics, saying "It works in all practical cases, so ignore the extremes at very small sizes and very high speeds, they are faulty models". Of course we don't allow that in the sciences, so why should we in ethics?

4Dmytry13y

In the practical reasoning, "A" is a shorthand for "I think A is true", et cetera - no absolute knowledge, nonzero false positive rate, and sufficiently refined moral theory has to take this into account. Just as thought experiment relying on e.g. absolute simultaneity would render itself irrelevant to special or general relativity, so does trolley problem's implicit assumption of absolute, reliable knowledge render it irrelevant to the extreme cases where the probability of event is much smaller than false positive rate.

0fubarobfusco13y

The analogy between moral theories and physics seems to suggest that just as we expect modern physics to act like Newtonian physics when dealing with big slow objects, we should expect some modern moral theory to act like folk morality when dealing with ordinary human life situations. Does that hold?

[-]latanius13y40

similar behaviours which we intuitively know we could live a fair lot better without

The situations presented are indeed such ones that we could live better without, but the whole point of thought experiments is to construct the worst possible world, and find a way to decide that works even under those circumstances. By your logic, we could easily end up saying "it's useless to argue about how one or two electrons behave, real world objects have much more of them... and anyway, tunneling effects are so weird and unintuitive that we surely have a wr... (read more)

2Dmytry13y

Well, the thought experiments are precisely the experiments in what's intuitive; that's why they are thought experiments rather than real experiments. It indeed is fairly useless to argue how one or two electrons behave, you have to take a look in the real world and see. Likewise with the trolley problems; the implicit presumption is that they are not completely irrelevant to the real world, yet they are because they neglect the false positive rate (assuming it to be 0) while discussing an extremely low probability event (whose probability is well below any plausible false positive rate). This sort of thing is precisely why Bayesian reasoning is so important.

[-]adamisom13y20

The issues brought up here impinge upon a concern of my own regarding issues that many Less Wrongers accept.

I don't know if I can let myself invest belief in things such as the Singularity (after I figure out what I mean) or cryonics without working some calculations out for myself.

The problem is that I don't know how to do that; and moreover, if I did, I'm worried that trying to do that would have so many possible points of error (and things to overlook) as to invariably give me overconfidence in whatever I was already leaning towards, with an anchoring bias towards acceptance given the posts on LW. Does anyone have any thoughts on this?

1Shmi13y

To me singularity and cryonics are two different beasts. The singularity is too far into the fog of the future to be taken seriously today. Maybe when there are some significant advances in the relevant fields, whenever that might happen. So far, even a single neuron is not fully simulated. The progress might go so many different ways, I find the singularity being just one of the many possible directions, and I know that my imagination is not up to par to even consider most of them. On the other hand, cryonics is basically a bet that a frozen brain can potentially be fully restored. We already know that single cells can be (sperm banks do it on an industrial scale), and there is some success with organs and even organisms. So we might be only a few steps away from being able to repair the damage done by freezing.

3adamisom13y

I think this may have been given a down vote because to some a Singularity seems settled and rational. I take it to be the idea that eventually--inevitably--machines will become smart enough to improve themselves, that once they become just a bit smarter than human programmers and with the capacity of self-modification, that it will be a runaway phenomenon. And that's why FAI is such a big deal: precisely because it's "far into the fog of future" in the sense that we don't have any control on the outcome (unless we align its values very precisely with our own). But here's the rub - for me, anyway - might it be rational to take your own reasoning as suspect about such 'foggy' matters? That is, even if you are doing your best to reason rationally, might it be best to assume that your potential knowledge is reduced to very little and the remainder filled in by cognitive bias? Okay, maybe this is a better approach---since questions like Singularity&FAI are potentially so important---can anyone point me to a resource that convinced them of its reality?

1Shmi13y

I would add, a resource that does not rely on the teachings of the local prophet, since most of us have already read those.

[-]atucker13y00

I like the first argument more than the second.

"How do we know that not donating to this one poor child with a rare cancer will actually save a bunch of Africans?" would be justified, even though if you're going for lives saved you really should just go with the Africans. Now, it helps that we have metrics, but you still have to decide to do something ever.

[-][anonymous]13y00

This is an excellent point, but wouldn't making a model as transparent as possible require an attention to one's audience? If so, wouldn't that preclude any method or rule about how to make a model transparent?

[-]beoShaffer13y00

Is the following a loosely accurate summary: "There are certain types of arguments where the probability of the argument not having any obvious flaws even if it is wrong is high enough that one shouldn't count the arguments as significant evidence for their conclusions."

[-]Nominull13y00

Who are you calling an "entirely unintelligent machine"? I think I'm offended.

0Dmytry13y

Computer running automatic theorem prover ;)

[-]Shmi13y00

One approach is to apply the scientific method: every model must have testable predictions. For the case of Pascal's mugging, you can ask the mugger to demonstrate their "magic powers from outside the Matrix" in a benign but convincing way ("Please show me a Turing machine that simulates an amoeba"). If they refuse or are unable to, you move on.

4khafra13y

The point of Pascal's Mugging dealing with a tiny probability of a really big harm vs. a high probability of a very small harm. If your mugger makes testable predictions about his power to carry out the really big harm, that turns the tiny probability into a reasonably large probability, and makes it a non-Pascalian hostage situation.

0Shmi13y

When confronted with highly speculative claims so beloved by philosophers, string theorists and certain AI apologists, my battle cry is "testable predictions!". If one argues in favor of a model that predicts a tiny probability of a really big harm, they better provide a testable justification of that model. In the case of Pascal's mugging, I have suggested a simple way to test if the model should be taken seriously. Such a test would have to be constructed specifically for each individual model, of course. If all you say is "I can't prove anything, but if I'm right, it'll be really bad", I yawn and move on.

4khafra13y

This is the normal response, even here at LW--I think there's a popular misperception that LW doctrine is to give the Pascal's Mugger money. The point of the exercise is to examine the thought processes behind that intuitive, obviously correct "no," when it appears, on the surface, to be the lower expected utility option. After all, we don't want to build an AI that can be victimized by Pascalian muggers. One popular option is the one you picked: Simply ignore probabilities below a certain threshhold, whatever the payoff. Another is to discount by the algorithmic complexity, or by the "measure" of the hostages. Yet another is to observe that, if 3^^^^3 people exist, a random person' (your) chances of being able to affect all the rest in a life-and-death way has to be scaled by 1/3^^^^3. Yet another is that, in a world where things like this happen, a dollar has near-infinite utility. Komponisto suggested that the kolmogorov complexity of 3^^^^3 deaths, or units of disutility, is much higher than that of the number 3^^^^3; so any such problem is inherently broken. Of course, if you're not planning to build an optimizing agent, your "yawn and move on" response is fine. That's what the problem is about, not signing up for cryonics or donating to SI or whatever (the proponents of the last two argue for relatively large probabilities of extremely large utilities).

2thomblake13y

To possibly expand on khafra's point, Pascal's Mugging is basically a computational problem. An expected utility maximizer, given a reward on the order of 3^^^3, would need a probability of less than 1/3^^^3 to discount it. But given limited computational resources and limited knowledge, there isn't an obvious algorithm that both calculates expected utility properly in normal cases, and actually arrives at a probability of 1/3^^^3 in the case of Pascal's Mugging (one might need more information than is in the universe to justify a probability that small).

0Shmi13y

In the "least convenient possible world" one would reduce the number of people to something that does not stretch the computational limits. I believe that my argument still holds in that case.

0thomblake13y

I'm confused. Is your invocation of LCPW supposed to indicate something that would solve this particular decision theory problem? If so, can you provide an algorithm that will successfully maximize expected utility in general and not fail in the case of problems like Pascal's Mugging?

0Shmi13y

Since 3^^^3 is unfeasible, suppose the mugger claims to simulate and kill "only" a quadrillion humans. The number is still large enough to overload one's utility, if you assign any credence to the claim. I am no expert in decision theory, but regardless of the exact claim, if the dude refuses to credibly simulate an amoeba, your decision is simple: ignore and move on. Please feel free to provide an example of Pascal mugging where this approach (extraordinary claims require extraordinary evidence) fails.

4Nornagest13y

Pascal's mugging only works if after some point your estimated prior for someone's ability to cause utilitarian losses of size n decreases more slowly than n increases; otherwise, claims of extravagant consequences make the mugging less likely to succeed as they grow more extravagant. "Magic powers from outside the Matrix" fill that role in the canonical presentation, since while the probability of that sort of magic existing is undoubtedly extremely small we don't have any good indirect ways of estimating its probability relative to its utilitarian implications, and we can't calculate it directly for the reasons thomblake gave a few comments up. A quadrillion humans, however, don't fit the bill. We can arrive at a reasonable estimate for what it'd take to run that kind of simulation, and we can certainly calculate probabilities that small by fairly conventional means: there's a constant factor here that I have no idea how to estimate, but 1 * 10^-15 is only about eight sigma from the mean on a standard normal distribution if I got some back-of-the-envelope math right. I'd feel quite comfortable rejecting a mugging of that form as having too little expected damage to be worth my time.

0Shmi13y

I must be missing something. To me a large number that does not require more processing power/complexity than the universe can provide is still large enough. TBH, even 10^15 looks to me too large to care, either the mugger can provide reasonable evidence or not, that's all that matters.

2Nornagest13y

If the mugger can provide reasonable evidence of his claims, it's not a decision-theoretically interesting problem; instead it becomes a straightforward, if exotic, threat. If the claim's modest enough that we can compute its probability by standard means, it becomes perfectly normal uncreditable rambling and stops being interesting from the other direction. It's only interesting because of the particular interaction between our means of updating probability values and a threat so fantastically huge that the expected loss attached to it can't be updated into neutral or negative territory by observation.

0Shmi13y

I guess that makes some philosophical sense. Not connected to any real-life decision making, though.

1thomblake13y

The problem was brought up in the context of making a computer program that correctly maximizes expected utility in all cases. Yes, in "real life" you can just ignore the mugger, but I don't know of a rigorous way of proving that's rational - your ability to ignore the mugger might well be a case of you getting the answer wrong, despite it seeming intuitively correct. If you think you have a definitive solution, please show your work, in math.

-3Shmi13y

Irrelevant, because the original thread started with my reply to: to which I pointed out that it is not rational to simply accept any argument that does not appear fallacious, not in the way EY defines rationality (as winning). If you apply the maxim "extraordinary claims require extraordinary evidence" (e.g. requesting to show at least a simulated amoeba before you consider the mugger's claims of simulating people any further), you win whether the mugger bluffs or not. WIN!

0Desrtopa13y

You can assign credence to the claim and still assign little enough that a quadrillion humans won't overload it. I think the claim the be able to simulate a quadrillion humans is a lot more probable than the claim to be able to simulate 3^^^3 (you'd need technology that almost certainly doesn't exist, but not outside-the-Matrix powers,) but I'd still rate it as being so improbable as to only account for a tiny fraction of an expected death.

0Shmi13y

I'm settling for just one quadrillion to avoid dealing with the contingency of "3^^^3 is impossible because complexity". The requirement of testability is not affected by the contingency.

0Desrtopa13y

If you assign the threat a probability of, say, 10^-20, the mugger is extorting considerably more dead children from you than you should expect to die if you don't comply.

-5Shmi13y

0gwern13y

And in the least convenient worlds?

0Shmi13y

Which contingent fact X do you mean?

0gwern13y

That you can demand testing in many real world scenarios, a heuristic not always usable. Or do you have a principled decision theory in mind, where testing is a key modification to the equations of expected-value etc and which defuses the mugging?

0Shmi13y

As a natural scientist, I would refuse to accept untestable models. Feel free to point out where this fails in any scenario that matters.

0dlthomas13y

How do you determine if the model is testable? What if there is in principle a test, but it has unacceptable consequences in at least one reasonably probable model?

0Shmi13y

For the particular scenario described in the Pascal's mugger, I provided a reasonable way to test it. If the mugger wants to dicker about the ways of testing it, I might decide to listen. It is up to the mugger to provide a satisfactory test. Hand-waving and threats are not tests. You are saying that there are models where testing is unfeasible or too dangerous to try. Name one.

1dlthomas13y

That such models exist is trivial - take model A, add a single difference B, where exercising the difference is bad. For instance, Model A: universe is a simulation Model B: universe is a simulation with a bug that will crash the system, destroying the universe, if X, but is otherwise identical to model A. Models that would deserve to be raised to the level of our attention in the first place, however, will take more thought.

0Shmi13y

By all means, apply more thought. Until then, I'm happy to stick by my testability assertion.

4dlthomas13y

A simple example might be if more of the worries around the LHC were a little better founded.

0Shmi13y

Ah yes, that is good one. Suppose a mad scientist threatens to make an earth-swallowing black hole in the LHC, unless his demands for world domination are met. What would be a prudent course of action? Calculate the utility of complying vs non-complying and go with the higher utility choice? Or do something else? (I have a solution or two, but will hold off suggesting any for now.)

0[comment deleted]13y

0TheOtherDave13y

Conversely, the inability of a putative Pascal's Mugger to make such predictions ought to apply a significant penalty to the plausibility of its claim. And simply increasing the threatened disutility won't necessarily help, since the more powerful the entity claims to be, the greater should be the implausibility of its inability to make testable predictions.

2Solvent13y

Well, that sometimes makes it more awkward to design the thought experiment. See the Least Convenient Possible World.

2Dmytry13y

The point would be that model has to be easily falsifiable - you should be able to see if it doesn't simulate amoeba. The Pascal's mugging is not easily falsifiable. I think of myself as a fairly intelligent individual; it took me 4 days to think that - whoah, the number of possible beings is not as huge as 3^^^^^3 ; if a being is made of 10^30 components each of which can be in either of 10^30 states, the number is only (10^30)^(10^30) which seems way smaller (not sure though, the knuth's up arrow is unwieldy to compare to anything). And that's some seriously frigging complex being I'm talking of (to human as human is to amoeba). I can't rely on myself to falsify such stuff reliably.

-2Shmi13y

You don't need to, as they would most likely be unable to simulate an amoeba, so you can call their bluff right there.

2Dmytry13y

The issue with pascal's mugging is that the refusal to simulate amoeba only decreases the probability of validity of the claim by a constant factor, and the mugger can stack more uparrows. Suppose that one in ten thousands superbeings refuses to demonstrate to you the simulated amoeba despite being capable of doing so. Also, what if it simulated amoeba but claims it'll torture 3^^^^^3 humans? I can falsify the latter part because there can not be this many distinct human mind states, but only after taking a lot of time to think about the claim.

2TheOtherDave13y

It seems reasonable to me that my confidence that the mugger can do what it claims to should be inversely proportional to the discrepancy between what the mugger claims and what the mugger is willing to demonstrate. Why do you say it should be a constant factor?

0Dmytry13y

What if one in ten thousands genuine super-beings is not a nice guy and refuses to demonstrate you stuff? The refusal is only linked to ability for those unable to demonstrate, or those barely able to demonstrate. If you mail world's top mathematicians a question what is 2*3+4 , very few will write you back; you may actually get higher willingness to 'demonstrate' ability to multiply and add from the elementary school kids.

0TheOtherDave13y

True. OTOH, very few of them will approach me with a "mathematician's mugging," either.

0fubarobfusco13y

Agreed. Humans already implement this: if someone keeps making bigger and bigger unproven claims, eventually they get a reputation as a braggart and a liar, and their claims don't count for anything.

0prase13y

When shown that model, how would you judge whether it really simulates an amoeba? It's fairly easy to create a program whose output resembles an amoeba on surface without simulating the movement of all its molecules.

2[anonymous]13y

It occurs to me that I may have made my answer to your post a little long,but I figured that when I was unsure of how to judge something, more tests are better than fewer tests. Starting with the basic tests from Wikipedia: 1: Does the model have organelles? 2: Does the model have cytoplasm? 3: Are the models organelles and cytoplasm enclosed by a cell membrane? 4: Does the model obtain its food through phagocytosis? 5: Is it a Heterotroph? 6: Does it have a single large tubular pseudopod at the anterior end, and several secondary ones branching to the sides? It would probably be reasonable for a fake amoeba simulation to have all of these. Then you start applying more tricky bits, like: 7: "Amoeba's are supposed to be able to recover from a forcible division if the nucleus is intact. If I remove some of the cytoplasm and keep the nucleus intact, does the simulation recover?" 8: "Cytoplasm is supposed to contain water. Does the bit of cytoplasm I extracted contain water?" 9: "Does the water divide into Hydrogen and Oxygen via electrolysis?" Also, be sure to test things where the answer is expected to be no, as well, in case the simulation is just programmed to spit out "Yes." repeatedly. 10: "Does the Amoeba contain multiple nuclei?" 11: "Does the Amoeba have a human nervous system?" 12: "Is the Amoeba more than 99% Salt?" Then ask 3 more questions which are LIKE question 7-12, but which I am not writing down, in case the person simply Googled for "Questions people have ever asked about Amoebas." and recorded appropriate answers. Now make sure to ask questions which aren't Yes/No questions, because Yes/No questions don't actually accumulate bits of evidence all that quickly. For instance 16: "How many Base pairs does the Ameoba have in it's genome?" 17: "How long is the Amoeba while moving?" 18: "How long is the Amoeba while not moving?" 19: "How fast can the Amoeba move?" 20: "Amoebas contain water. Water can put out forest fires. How many o

0Shmi13y

Thank you, you have illustrated my point that the test to even consider the mugger seriously, let alone take his claims at face value is easy to devise, but hard to pass, unless one is indeed an expert in simulating living things very convincingly.

0Dmytry13y

But there is nonzero probability that the genuine being outside the matrix would not be able to provide satisfactory demonstration due to e.g. limited bandwidth channel it may have to our universe (suppose that someone else is in charge of our universe). (If that being is omnipotent in our universe you can just ask it to show the flaming text in front of your eyes, no need to simulate amoebas. But if it can demonstrate this, then it is straightforwardly a God and there's no paradox or question of any kind)

2Shmi13y

Or a capable trickster. Which one do you think is more likely?

0Dmytry13y

Say, I were to make a flaming text appear right in the middle of your visual field, moving with your head as you turn it around, like right as you're reading this sentence. Most likely explanation would be that you are hallucinating, at which point you can just as well give up and lie still for some hours hoping for image to go away.

Moderation Log