I think it's cleaner to build the foundation around the basic laws of physics. You don't need the laws of reasoning in the description of physics, they define the way reality flows. Leaving the margin for doubt on the laws themselves, all the higher-level things like minds and principles of reasoning within the current framework can be built on top of that, like special kind of physical structures with a certain kind of dynamics. I think about the goal-driven intelligence (optimization process) as an attractor in the dynamics of reality, which drives it towards a certain kind of dynamics. When morality of the intelligent agent is changed, either by being broken significantly or by being influenced, it means that the high-level description that fits the process that is reality is changed. Where before the dynamics looked like it was heading in this direction, now it jumped to a form where it's heading in another direction, or is changed the course, so that it's heading in a slightly different way. The laws of physics are the substrate, on which one has to figure a way of building the attractor.
Another question is that if the mind is running inside a simulation, it is not semantically bound to the structure of real world, its goal gets hijacked by substituting the reality by the simulated happy button, the limited simulated reality that is accessible to inspection. Such mind can't have a global attractor effect on the flow of reality, but it can bind it locally, just as an ordinary human optimizer, so I guess it's not fundamentally different.
Followup to: The Genetic Fallacy, Where Recursive Justification Hits Bottom
Yesterday I said that—because humans are not perfect Bayesians—the genetic fallacy is not entirely a fallacy; when new suspicion is cast on one of your fundamental sources, you really should doubt all the branches and leaves of that root, even if they seem to have accumulated new evidence in the meanwhile.
This is one of the most difficult techniques of rationality (on which I will separately post, one of these days). Descartes, setting out to "doubt, insofar as possible, all things", ended up trying to prove the existence of God—which, if he wasn't a secret atheist trying to avoid getting burned at the stake, is pretty pathetic. It is hard to doubt an idea to which we are deeply attached; our mind naturally reaches for cached thoughts and rehearsed arguments.
But today's post concerns a different kind of difficulty—the case where the doubt is so deep, of a source so fundamental, that you can't make a true fresh beginning.
Case in point: Remember when, in the The Matrix, Morpheus told Neo that the machines were harvesting the body heat of humans for energy, and liquefying the dead to feed to babies? I suppose you thought something like, "Hey! That violates the second law of thermodynamics."
Well, it does violate the second law of thermodynamics. But if the Matrix's makers had cared about the flaw once it was pointed out to them, they could have fixed the plot hole in any of the sequels, in fifteen seconds, this easily:
Now, mind you, I am not saying that this excuses the original mistake in the script. When my mind generated this excuse, it came clearly labeled with that warning sign of which I have spoken, "Tada! Your mind can generate an excuse for anything!" You do not need to tell me that my plot-hole-patch is a nitwit idea, I am well aware of that...
...but, in point of fact, if you woke up out of a virtual reality pod one day, you would have to suspect all the physics you knew. Even if you looked down and saw that you had hands, you couldn't rely on there being blood and bone inside them. Even if you looked up and saw stars, you couldn't rely on their being trillions of miles away. And even if you found yourself thinking, you couldn't rely on your head containing a brain.
You could still try to doubt, even so. You could do your best to unwind your thoughts past every lesson in school, every science paper read, every sensory experience, every math proof whose seeming approval by other mathematicians might have been choreographed to conceal a subtle flaw...
But suppose you discovered that you were a computer program and that the Dark Lords of the Matrix were actively tampering with your thoughts.
Well... in that scenario, you're pretty much screwed, I'd have to say.
Descartes vastly underestimated the powers of an infinitely powerful deceiving demon when he supposed he could trust "I think therefore I am." Maybe that's just what they want you to think. Maybe they just inserted that conclusion into your mind with a memory of it seeming to have an irrefutable chain of logical support, along with some peer pressure to label it "unquestionable" just like all your friends.
(Personally, I don't trust "I think therefore I am" even in real life, since it contains a term "am" whose meaning I find confusing, and I've learned to spread my confidence intervals very widely in the presence of basic confusion. As for absolute certainty, don't be silly.)
Every memory of justification could be faked. Every feeling of support could be artificially induced. Modus ponens could be a lie. Your concept of "rational justification"—not just your specific concept, but your notion that any such thing exists at all—could have been manufactured to mislead you. Your trust in Reason itself could have been inculcated to throw you off the trail.
So you might as well not think about the possibility that you're a brain with choreographed thoughts, because there's nothing you can do about it...
Unless, of course, that's what they want you to think.
Past a certain level of doubt, it's not possible to start over fresh. There's nothing you can unassume to find some firm rock on which to stand. You cannot unwind yourself into a perfectly empty and perfectly reliable ghost in the machine.
This level of meta-suspicion should be a rare occasion. For example, suspecting that all academic science is an organized conspiracy, should not run into anything like these meta-difficulties. Certainly, someone does not get to plead that unwinding past the Bible is impossible because it is too foundational; atheists walk the Earth without falling into comas. Remember, when Descartes tried to outwit an infinitely powerful deceiving demon, he first tried to make himself absolutely certain of a highly confusing statement, and then proved the existence of God. Consider that a caution about what you try to claim is "too basic for a fresh beginning". And even basic things can still be doubted, it is only that we use our untrustworthy brains to doubt them.
Or consider the case of our existence as evolved brains. Natural selection isn't trustworthy, and we have specific reason to suspect it. We know that evolution is stupid. We know many specific ways in which our human brains fail, taken beyond the savanna. But you can't clear your mind of evolutionary influences and start over. It would be like deciding that you don't trust neurons, so you're going to clear your mind of brains.
And evolution certainly gets a chance to influence every single thought that runs through your mind! It is the very reason why you exist as a thinker, rather than a lump of carbon—and that doesn't mean evolution summoned a ghost-in-the-machine into you; it designed the ghost. If you learn culture, it is because you were built to learn culture.
But in fact, we don't run into unmanageable meta-trouble in trying to come up with specific patches for specific known evolved biases. And evolution is stupid, so even though it has set up self-deceptive circuits in us, these circuits are not infinitely difficult to comprehend and outwit.
Or so it seems! But it really does seem that way, on reflection.
There is no button you can press to rewind past your noisy brain, and become a perfectly reliable ghost of perfect emptiness. That's not just because your brain is you. It's also because you can't unassume things like modus ponens or belief updating. You can unassume them as explicit premises for deliberate reasoning—a hunter-gatherer has no explicit concept of modus ponens—but you can't delete the actual dynamics (and all their products!)
So, in the end, I think we must allow the use of brains to think about thinking; and the use of evolved brains to think about evolution; and the use of inductive brains to think about induction; and the use of brains with an Occam prior to think about whether the universe appears to be simple; for these things we really cannot unwind entirely, even when we have reason to distrust them. Strange loops through the meta level, I think, are not the same as circular logic.
Part of The Metaethics Sequence
Next post: "Rebelling Within Nature"
Previous post: "My Kind of Reflection"