Well Eliezer was found of italicizing words in his text, doesn't provide references for most of his statements and wrote quit a few walls of text. I mean the sequences are huge.

I don't think it requires a separate explanation.

Suppose I have a belief that I arrived at via some process other than evaluating evidence. Suppose further that I find that belief satisfying in some way... I prefer believing it to not-believing it.

If I suspect that evaluating available evidence will weaken my belief, that gives me a reason to avoid evaluating that evidence.

Having decided to avoid evaluating that evidence, I have incentive to believe that the evidence is worthless.

Because the outside view frequently gives better predictions than the inside view. For example as Manfred recently noted in a different thread, Mike Darwin's predictions were

[m]uch more accurate outside of his specialty.

Religious and other dogmas need not make sense. Indeed, they may work better if they are not logical. Logical and useful ideas pop-up independently and spread easily, and widely accepted ideas are not very good badges. You need a unique idea to identify your group. It helps to have a somewhat costly idea as a dogma, because they are hard to fake and hard to deny. People would need to invest in these bad ideas, so they would be less likely to leave the group and confront the sunk cost. Also, it's harder to deny allegiance to the group afterwards, because no one in their right minds would accept an idea that bad for any other reason.

If you have a naive interpretation of the dogma, which regards it as an objective statement about the world, you will tend to question it. When you’re contesting the dogma, people won’t judge your argument on its merits: they will look at it as an in-group power struggle. Either you want to install your own dogma, which makes you a pretender, or you’re accepted a competing dogma, which makes you a traitor. Even if they accept that you just don’t want to yield to the authority behind the dogma, that makes you a rebel. Dogmas are just off-limits to criticism.

Public display of dismissive attitude to your questioning is also important. Taking it into consideration is in itself a form of treason, as it is interpreted as entertaining the option of joining you against the authority. So it’s best to dismiss the heresy quickly and loudly, without thinking about it.

My heretical by LW standards/scary/worst possible world idea on this is that society needs such dogma. It needs it badly, because coordination is hard. Weak evidence in this direction is that no society ever seems to have existed without it.

That's not the scary part. The scary part is that screwy metaphysical entities like say a God here or there or Reincarnation may in fact impose lower costs on a society than a dogmatic adherence to a particular interpretation of say "justice" or "fatherland" or "the dictatorship of the proletariat".

It would seriously suck to live in that world.

Fortunately, being a rock star from Mars, I live in the world of happy dust and goddesses.

But the probability that a true random generator will output 0000000000 should be the same as the probability that it will output 0010111101, because all sequences of equal length are equally likely.

With a fair random generator:

p(0000000000) = 1/2^10
p(1000000000) = 1/2^10
p(0100000000) = 1/2^10
p(0010000000) = 1/2^10

The numbers produced are independent of each other and for our purposes we don't care about the order. The relevant thing is how likely it is is to produce a given total number of zeroes or ones.

p(just one 1) = 10/2^10; A whole heap more likely!

So the chance that the generator is fair is rather slim. You can calculate just how slim by simply applying bayes rule (and doing some integration).

On a related note if you role two six sided dice you are just as likely to get two sixes as you are to get a three and a five. But if you are playing Settlers of Catan and put all your settlements next to the twelve instead of the eight then you are probably going to lose.

I see a possible confusion in your post, in the sense that you have apparently fused two hypothesis into one: *A is random *A is fair

The two are not equivalent: A can be a random generator with p(0)!=50% or it can have p(0)=50% but output a very predictable string, e.g. 01010101010101010101...

Going back to your example if you hypothesis is A=(random et fair) then seeing the string 0000000000 can lower your belief in the statement (not for the "random" part, but for the "fair" part"). If your hypothesis was A=(random et p(0)= 0.00000000000000001%) then the string could raise your belief instead.

The biased random generator is also just as likely to output 0000000000 as it is 0010111101.

This is the mistake.

If you actually do the maths the biased generator is significantly more likely to output 0000000000 than 0010111101.

For a much simpler example, suppose we run on two times. The random generator outputs 00 25% of the time, 01 25% of the time, 10 25% of the time and 11 25% of the time.

For the biased generator, we need calculus. First suppose its p(0) = x. Then p(00 | p(0) = x ) = x^2. Since we have what is essentially a uniform distribution over [0,1] (the presence of absence of a single point makes no difference) we need to integrate f(x) = x^2 over the interval [0,1], which gives and answer of p(00) = 1/3. The same method gives p(11) = 1/3 and p(01) = p(10) = 1/6.

The general rule, is that if we run it n times, for any k between 0 and n the chance of it outputting k 1s is 1/(n+1), and that probability is shared out evenly among all possible different ways of outputing k 1s (also derivable from calculus). Thus p(0000000000) = 9.1% while p(0010111101) = 0.043%.

Well -- there's also the human habit of skimming over text to extract the "useful" information -- especially in timed tests or where we believe the text extraneous to the actual function. Word problems are pretty much always an exercise in "these words are an obstacle between me and the formula". So it stands to reason -- superficially that is -- that making it harder to read the font (without increasing the difficulty of the language) would act as a "counterbalance" to the impetus to get done as quickly as possible with the verbal and on to the mathematical.

In other words; I'm asserting a hypothesis that this is illustrating an underlying mechanism regarding how test-takers handle reading their examination questions.

Is it "more rational" to spend more time on the exam question? Perhaps. (Almost definitely, since doing so increases their scores as shown here.) But then we have to ask what the actual goal of test-takers is at the time of taking the exam. Is it truly to "get the highest score"? Or is it "avoid the greatest amount of anxiety this exam produces in me {where 'me'='person taking the test'}"? Very often I have found the latter to be the case -- but I would suspect that this is hardly irrational; those individuals frequently aren't much invested in higher exam scores than are necessary to achieve a passing score. Getting the exam done quicker without falling below that score, then, is the rationally optimum resolution.

Hence my doubt as to whether it would be called a "rational" behavior over a "necessary" one.

For example, it could be expected that this lower-level burden would drain cognitive "energy" from higher functions trying to solve the mathematical problem.

In observing how people take tests, I've seen that people first 'extract' the information from the question and then move on to deriving its answer. Your point is valid, however.

Do you think that people could behave rationally with such a simple intervention?

A higher cognitive burden on comprehension tautologically requires more thought per unit of information occur.

I don't know if it's a question of rational behavior but of necessary.