Yesterday's post concluded:

To form accurate beliefs about something, you really do have to observe it. It's a very physical, very real process: any rational mind does "work" in the thermodynamic sense, not just the sense of mental effort...  So unless you can tell me which specific step in your argument violates the laws of physics by giving you true knowledge of the unseen, don't expect me to believe that a big, elaborate clever argument can do it either.

One of the chief morals of the mathematical analogy between thermodynamics and cognition is that the constraints of probability are inescapable; probability may be a "subjective state of belief", but the laws of probability are harder than steel.

People learn under the traditional school regimen that the teacher tells you certain things, and you must believe them and recite them back; but if a mere student suggests a belief, you do not have to obey it.  They map the domain of belief onto the domain of authority, and think that a certain belief is like an order that must be obeyed, but a probabilistic belief is like a mere suggestion.

They look at a lottery ticket, and say, "But you can't prove I won't win, right?"  Meaning:  "You may have calculated a low probability of winning, but since it is a probability, it's just a suggestion, and I am allowed to believe what I want."

Here's a little experiment:  Smash an egg on the floor.  The rule that says that the egg won't spontaneously reform and leap back into your hand is merely probabilistic.  A suggestion, if you will.  The laws of thermodynamics are probabilistic, so they can't really be laws, the way that "Thou shalt not murder" is a law... right?

So why not just ignore the suggestion?  Then the egg will unscramble itself... right?

It may help to think of it this way - if you still have some lingering intuition that uncertain beliefs are not authoritative:

In reality, there may be a very small chance that the egg spontaneously reforms.  But you cannot expect it to reform.  You must expect it to smash.  Your mandatory belief is that the egg's probability of spontaneous reformation is ~0.  Probabilities are not certainties, but the laws of probability are theorems.

If you doubt this, try dropping an egg on the floor a few decillion times, ignoring the thermodynamic suggestion and expecting it to spontaneously reassemble, and see what happens.  Probabilities may be subjective states of belief, but the laws governing them are stronger by far than steel.

I once knew a fellow who was convinced that his system of wheels and gears would produce reactionless thrust, and he had an Excel spreadsheet that would prove this - which of course he couldn't show us because he was still developing the system.  In classical mechanics, violating Conservation of Momentum is provably impossible.  So any Excel spreadsheet calculated according to the rules of classical mechanics must necessarily show that no reactionless thrust exists - unless your machine is complicated enough that you have made a mistake in the calculations.

And similarly, when half-trained or tenth-trained rationalists abandon their art and try to believe without evidence just this once, they often build vast edifices of justification, confusing themselves just enough to conceal the magical steps.

It can be quite a pain to nail down where the magic occurs - their structure of argument tends to morph and squirm away as you interrogate them.  But there's always some step where a tiny probability turns into a large one - where they try to believe without evidence - where they step into the unknown, thinking, "No one can prove me wrong".

Their foot naturally lands on thin air, for there is far more thin air than ground in the realms of Possibility.  Ah, but there is an (exponentially tiny) amount of ground in Possibility, and you do have an (exponentially tiny) probability of hitting it by luck, so maybe this time, your foot will land in the right place!  It is merely a probability, so it must be merely a suggestion.

The exact state of a glass of boiling-hot water may be unknown to you - indeed, your ignorance of its exact state is what makes the molecules' kinetic energy "heat", rather than work waiting to be extracted like the momentum of a spinning flywheel.  So the water might cool down your hand instead of heating it up, with probability ~0.

Decide to ignore the laws of thermodynamics and stick your hand in anyway, and you'll get burned.

"But you don't know that!"

I don't know it with certainty, but it is mandatory that I expect it to happen.  Probabilities are not logical truths, but the laws of probability are.

"But what if I guess the state of the boiling water, and I happen to guess correctly?"

Your chance of guessing correctly by luck, is even less than the chance of the boiling water cooling your hand by luck.

"But you can't prove I won't guess correctly."

I can (indeed, must) assign extremely low probability to it.

"That's not the same as certainty, though."

Hey, maybe if you add enough wheels and gears to your argument, it'll turn warm water into electricity and ice cubes!  Or, rather, you will no longer see why this couldn't be the case.

"Right! I can't see why couldn't be the case!  So maybe it is!"

Another gear?  That just makes your machine even less efficient.  It wasn't a perpetual motion machine before, and each extra gear you add makes it even less efficient than that.

Each extra detail in your argument necessarily decreases the joint probability.  The probability that you've violated the Second Law of Thermodynamics without knowing exactly how, by guessing the exact state of boiling water without evidence, so that you can stick your finger in without getting burned, is, necessarily, even less than the probability of sticking in your finger into boiling water without getting burned.

I say all this, because people really do construct these huge edifices of argument in the course of believing without evidence.  One must learn to see this as analogous to all the wheels and gears that fellow added onto his reactionless drive, until he finally collected enough complications to make a mistake in his Excel spreadsheet.