In “What is Evidence?” I wrote:1
This is why rationalists put such a heavy premium on the paradoxical-seeming claim that a belief is only really worthwhile if you could, in principle, be persuaded to believe otherwise. If your retina ended up in the same state regardless of what light entered it, you would be blind . . . Hence the phrase, “blind faith.” If what you believe doesn’t depend on what you see, you’ve been blinded as effectively as by poking out your eyeballs.
Cihan Baran replied:2
I can not conceive of a situation that would make 2 + 2 = 4 false. Perhaps for that reason, my belief in 2 + 2 = 4 is unconditional.
I admit, I cannot conceive of a “situation” that would make 2 + 2 = 4 false. (There are redefinitions, but those are not “situations,” and then you’re no longer talking about 2, 4, =, or +.) But that doesn’t make my belief unconditional. I find it quite easy to imagine a situation which would convince me that 2 + 2 = 3.
Suppose I got up one morning, and took out two earplugs, and set them down next to two other earplugs on my nighttable, and noticed that there were now three earplugs, without any earplugs having appeared or disappeared—in contrast to my stored memory that 2 + 2 was supposed to equal 4. Moreover, when I visualized the process in my own mind, it seemed that making xx and xx come out to xxxx required an extra x to appear from nowhere, and was, moreover, inconsistent with other arithmetic I visualized, since subtracting xx from xxx left xx, but subtracting xx from xxxx left xxx. This would conflict with my stored memory that 3 - 2 = 1, but memory would be absurd in the face of physical and mental confirmation that xxx - xx = xx.
I would also check a pocket calculator, Google, and perhaps my copy of 1984 where Winston writes that “Freedom is the freedom to say two plus two equals three.” All of these would naturally show that the rest of the world agreed with my current visualization, and disagreed with my memory, that 2 + 2 = 3.
How could I possibly have ever been so deluded as to believe that 2 + 2 = 4? Two explanations would come to mind: First, a neurological fault (possibly caused by a sneeze) had made all the additive sums in my stored memory go up by one. Second, someone was messing with me, by hypnosis or by my being a computer simulation. In the second case, I would think it more likely that they had messed with my arithmetic recall than that 2 + 2 actually equalled 4. Neither of these plausible-sounding explanations would prevent me from noticing that I was very, very, very confused.3
What would convince me that 2 + 2 = 3, in other words, is exactly the same kind of evidence that currently convinces me that 2 + 2 = 4: The evidential crossfire of physical observation, mental visualization, and social agreement.
There was a time when I had no idea that 2 + 2 = 4. I did not arrive at this new belief by random processes—then there would have been no particular reason for my brain to end up storing “2 + 2 = 4” instead of “2 + 2 = 7.” The fact that my brain stores an answer surprisingly similar to what happens when I lay down two earplugs alongside two earplugs, calls forth an explanation of what entanglement produces this strange mirroring of mind and reality.
There’s really only two possibilities, for a belief of fact—either the belief got there via a mind-reality entangling process, or not. If not, the belief can’t be correct except by coincidence. For beliefs with the slightest shred of internal complexity (requiring a computer program of more than 10 bits to simulate), the space of possibilities is large enough that coincidence vanishes.4
Unconditional facts are not the same as unconditional beliefs. If entangled evidence convinces me that a fact is unconditional, this doesn’t mean I always believed in the fact without need of entangled evidence.
I believe that 2 + 2 = 4, and I find it quite easy to conceive of a situation which would convince me that 2 + 2 = 3. Namely, the same sort of situation that currently convinces me that 2 + 2 = 4. Thus I do not fear that I am a victim of blind faith.5
1See Map and Territory.
2Comment: http://lesswrong.com/lw/jl/what_is_evidence/f7h.
3See “Your Strength as a Rationalist” in Map and Territory.
4For more on belief formation and beliefs of fact, see “Feeling Rational” and “What Is Evidence?” in Map and Territory. For more on belief complexity, see “Occam’s Razor” in the same volume.
5If there are any Christians reading this who know Bayes’s Theorem, might I inquire of you what situation would convince you of the truth of Islam? Presumably it would be the same sort of situation causally responsible for producing your current belief in Christianity: We would push you screaming out of the uterus of a Muslim woman, and have you raised by Muslim parents who continually told you that it is good to believe unconditionally in Islam.
Or is there more to it than that? If so, what situation would convince you of Islam, or at least, non-Christianity? And how confident are you that the general kinds of evidence and reasoning you appeal to would have been enough to dissuade you of your religion if you had been raised a Muslim?
I understand the point Eliezer's trying to make here. However, you (whoever's reading this) could not convince me that ss0 + ss0 =sss0 in Peano arithmetic (I define the scenario in which my mind is directly manipulated so that I happen to believe this not to constitute "convincing me"). Here's why I believe this position to be rational:
A)In order for me to make this argument, I have to presume communication of it. It's not that I believe the probability of that communication to be 1. Certainly many people might read this comment and not know Peano arithmetic, misunderstand my language, not finish reading, etc. etc. etc. and the probability of this is nontrivial. However, arguments are directed at the possible worlds in which they are understood.
B)Communication of "ss0 + ss0 =" as a statement of Peano arithmetic already fully constrains the answer to be "ssss0" simply by virtue of what these symbols mean. That is to say, that having understood these symbols and Peano arithmetic, no further experience is necessary to know that "sss0" is wrong. Mental flaws at any point in this process or understanding are possible, but they exist only within possible worlds in which communication of these ideas does not actually occur because to think that "ss0 + ss0 = sss0" is to misunderstand Peano arithmetic, and understanding Peano arithmetic is a prerequisite for understanding a claim about it.
Therefore
C)There is no possible world within which I can be convinced of the properly communicated concept "ss0 + ss0 = sss0". Of course, this doesn't mean there's no possible world in which I can be convinced that I am experiencing a neurological fault or being manipulated, or that there are no possible worlds in which I happen to wrongly believe that ss0 + ss0 = sss0. It's just that someone experiencing a neurological fault or being manipulated is not the same thing as someone being convinced.
A similar argument holds for the impossibility of me convincing myself that ss0 + ss0 = sss0. I understand ss0 + ss0 = ssss0 in Peano arithmetic well enough that I can review in a very short period of time why it must be so. Thus you would literally have to make me forget that I know this in order to have me believe otherwise, which hardly counts as "convincing." This does not mean that I am presuming mental errors or Dark Lords of the Matrix to be impossible. For clarification, here's what a runthrough of me experiencing what Eliezer proposes would look like:
Because that stored memory entails an understanding of why, I run through those reasons. If they're incomplete this constitutes me "forgetting that I know this." (It does not mean that I don't know this now. Right now I do.) Therefore I don't have a "stored memory that 2 + 2 was supposed to equal 4." I have an incomplete stored memory which tries to say something about 2, +, =, and 4 (if my personality were intact I would probably try and re-derive the missing parts of it, after calling 911). Either way I identify a cognitive fault. In real life waking up to this my most likely suspect would be that my experience of one earplug disappearing was deleted before I processed it, but there are lots of other possibilities as well. If I repeated the experiment multiple times I would consider either a systematic fault or "being messed with" at a fundamental level.
Still presuming an intact line of reasoning saying why this must not be so, I would again identify a cognitive fault, and a pretty cool one at that. Something this intricate might well leave me suspecting Dark Lords of the Matrix as a nontrivial possibility, provided all other cognitive functions seemed fully intact. Still wouldn't be as likely as a weird brain fault, though. I would definitely have fun investigating this.
Dark Lords of the Matrix bump higher, but Psychosis has definitely leapt into the front of the pack.
I could keep going, of course. These last few presume I can still reason out something like Peano arithmetic. If I can't incidentally, then of course they look different but I still don't think it would be accurate to describe any possible outcome as "me being convinced that 2 + 2 = 3." If you run all the way down the list until you literally delete all things that I know and all ways I might obtain them, I would describe that as a possible universe in which "me" has been deleted. The strict lower bound on where I can still stumble across my cognitive fault and/or manipulation is when my reasoning ability is no longer Turing complete. This essentially requires the elimination of all complex thought, though of course making it merely unlikely for me to stumble upon the fault is much easier--just delete everything I know about formal mathematics, for example.
tl;dr
I agree with most of what Eliezer is saying, but wouldn't say that I could be convinced 2 + 2 = 3. Does this make my belief unconditional? Dependant on me understanding what 2 + 2 = 3 means, maybe it does. Maybe an understanding of 2, +, =, 3, and 4 necessitates 2 + 2 = 4 for a rational mind, and any deviation from this, even in internal mental processes, would be identifiable as a fault. After all, you can run a software program to detect flaws in some computer processors.
Extrapolating from Eliezers line of reasoning you would probably find that although you remember ss0 + ss0 = ssss0, if you try to derive ss0 + ss0 from the peano axioms, you also discover it ends up as sss0, and starting with ss0 + ss0 = ssss0 quickly leads you to a contradiction.