This is a good post, and it suggests a whole series of similar posts: take each of the cues people treat as signs of rationality, and dig deeper to look for when exactly rational people would or would not display those signs. Watch out for people proud to display the cue even when rational people would not have it.

"- An unresolved doubt is a null-op.

• An uninvestigated doubt might as well not exist."

Perhaps we're just using words differently, but I'm not sure I agree with either of these. I would have thought that recognising valid doubts would be useful in making decisions, even when the information necessary to resolve such doubts with certainty may not be available; and in some cases, the gains in terms of improved decision-making may not be worth the cost of investigating and resolving the doubt.

I think I'm using "doubt" as almost coextensive with "uncertainty", and I'm not entirely sure what else it would mean, but do you mean something else?

A great post (in a series of great recent posts from Eliezer), and so far the comments on this post are very strong too.

PS I love this line for the double scoop of transparency: "Making a great public display of doubt to convince yourself that you are a rationalist, will do around as much good as wearing a lab coat."

Good point, conchis. By "doubt" I don't mean assigning a probability unequal to 1 - all probabilities are like that, in my book. If I'm pretty sure that a coinflip is fair, I don't say I "doubt" whether the result will be heads or tails - it doesn't feel the same as doubting whether it's possible to revive a cryonics patient.

It seems to me that the word "doubt" could refer to two different things. First, it could be descriptive, an emotion that human beings sometimes feel, for example what kids feel when they start to wonder whether Santa Claus really exists. Second, "doubt" could have a pure mathematical meaning: an ideal Bayesian seeing a probabilistic opportunity to destroy a belief (downgrade its probability) by following a path of investigation. A human rationalist's Type-1 'doubts' should also qualify as Type-2 'doubts'.

HA, what do you mean by "transparency"?

Eliezer,

http://www.mja.com.au/public/issues/174_07_020401/mvdw/mvdw.html

Particularly scary sentence:

"And yet, the practice of medicine involves more than its subservience to evidence or science. It also involves issues such as the meaning of service and feelings of professional pride."

Hopefully: Heh. I have sometimes thought that all professional lectures on rationality should be delivered while wearing a clown suit, to prevent the audience from confusing seriousness with solemnity.

But I still don't know what you mean by "transparency"; you've used it before, and I can think of more than possible meaning to it.

I think the phrasing that 'Jesuit novices are told to doubt everything' is a loose interpretation of the very demanding initial two year training period that novices have to undergo. The Jesuits want only the best people and try hard to weed out the weaker applicants, so that only the most dedicated survive the initial training period.

The Catholic Encyclopaedia describes the Jesuit training here: Note: Their article may be biased. :) http://www.newadvent.org/cathen/14081a.htm

Quote: the novice is trained diligently in the meditative study of the truths of religion, in the habit of self-knowledge, in the constant scrutiny of his motives and of the actions inspired by them, in the correction of every form of self-deceit, illusion, plausible pretext, and in the education of his will, particularly in making choice of what seems best after careful deliberation and without self-seeking. Deeds, not words, are insisted upon as proof of genuine service, and a mechanical, emotional, or fanciful piety is not tolerated.

If the applicant survives for two years, then his real training begins.

BillK

Eliezer, I'm using transparency to mean people wearing lab coats, or making great public displays of doubt being open and honest to themselves and others about why they're doing so. I think it's a standard usage of the word.

Eliezer, I still don't think the definition of doubt you posit in your comment accounts well for its usage in your post. There are a number of topics that I have only cursory knowledge on, and positions on ideas in those topics that I judge as having only a limited probability of truth. Most of these topics are on the periphery of my awareness most of the time, and only have a very limited usefulness to me, and are not very interesting. While I could easily investigate these matters in more depth and come to a more well-considered position on many ideas, I have no motivation to. So do I have doubts about these? According to your definition, yes, but that doesn't quite work. If the topics were to become important to me for some reason (whether because they were necessary to solve some problem I were working on or just because I became interested in them) then I would have doubts, and I would fill out my knowledge to get rid of them.

Maybe all your definition needs is a proviso that the utility of the increased certainty about a topic (which would be determined by the goal system of the intelligence, and might be technically arational) must outweigh the cost of the line of inquiry suggested by the doubt. More precisely, that the utility of resolving the doubt be higher than the utility of saving resources for other pursuits.

Eliezer, I think I'm starting to see what you're getting at a little more with your second definition ("an ideal Bayesian seeing a probabilistic opportunity to destroy a belief (downgrade its probability) by following a path of investigation"). But I'm still not entirely sure whether I would agree with the two points in your post that I originally took issue with.

If I come up with a reason to doubt the probability I previously assigned to some outcome, then, because (as an ideal Bayesian) I shouldn't expect to change the probability assigned to something as a result of new evidence, I should presumably revise my probability estimate down immediately - before seeing any evidence at all. But once that's done, whether or not the doubt warrants further investigation, or still needs in some sense to be resolved still seems an open issue. To be honest, I'm not even entirely sure what "resolution" would mean in this context any more. (Unless, perhaps, you simply mean the initial downgrading of probability?)

Conchis, the path of investigation would have a high probability of changing your current estimate up or down (by the same weighted amount), so your current estimate can still be your best guess as an ideal Bayesian while still making the investigation likely to change it to be more accurate.

pdf23ds: Of course, but I interpreted Eliezer as having in mind an asymmetrical (and, I might add, intuitive) definition of doubt, that placed a higher probability on downgrading. I might have misunderstood him though.

Conchis, I didn't get the impression that a doubt more often downgrades than it upgrades, since you can just as easily doubt a low estimation as a high one, and since you can express any probability of X greater than .5 as one of ~X under .5. Someone can just as easily doubt their atheism as they can their theism.

But the doubts could be asymmetrical. What about a path of investigation (POI) that had a 10% chance of increasing you estimation by .3 and a 90% chance of lowering it by .03? (Those numbers might not actually be balanced, but I think you get the idea.)

pdf23ds, I'm not sure we're really disagreeing about anything here. I would naturally define a doubt in exactly the way you seem to suggest. But if you use it that way, then the two points of Eliezer's I took issue with in my first comment don't seem to follow. I took Eliezer's response as attempting to find an alternative definition on which they did follow, and then pointed out that the alternative definition he seemed to be offering didn't make sense. Maybe I misunderstood something along the way here, but I'm certainly not arguing for that definition myself.

'No, they are to doubt that perhaps their doubts may grow and become stronger.'

This establishes the a rule for using doubt as bias against any future information that would perhaps increase preexisting doubts. It is a bias because it does not apply the rule for doubt about anything that perhaps increases belief, or perhaps maintains the assessment of current doubt/belief.

Having instructed a rule for doubt about 'perhaps' for only one 'perhaps', the rule provides a default increase in the amount of information required to overcome that particular 'perhaps'. That increased amount of information needed is the amount needed to overcome the imposed 'doubt'.

That this amount is established by rule conveys the requirement of 'faith' because it replaces a methodology that is falsifiable with one that is not.

Hi,

Interesting post, but I think you forgot to account for the time issue : « An uninvestigated doubt might as well not exist. » is true if the doubt stays uninvestigated forever. But if it is uninvestigated for now (even for a period of several years), the mere fact there is a doubt means the probability of investigating later on is higher than if there is no doubt at all (P(Investigate_in_10years) and P(Doubt_now) are not independent).

For example, the fact that right now I'm now doubtful about cyronics means that I may investigate the issue later on, with a higher probability that my friends who says "cryonics is non-sense".

So I would reformulate as :

• A doubt that stays uninvestigated forever might as well not exist

• An uninvestigated doubt is only useful as it may drive you to investigate in the future.

Making a great public display of doubt to convince yourself that you are a rationalist, will do around as much good as wearing a lab coat.

Are you sure it will do that much? Wearing a lab coat actually does increase perceptiveness (you can safely ignore everything before the experimental overview). If signaling rationality actually increases rationality, it may be useful after all. Considering that both the outward behaviors and the clothing operate through the same mechanism, namely associating oneself with that image, it's likely that a great public display of doubt is indeed as useful as wearing a lab coat.

instead of taking questions and asking them, take questions and identify the situations where the doubt arise, then avoid those curiosity generators for an alternative solution to curiosity stopping

Not sure if this is the proper place to say this but your first link is broken.

http://www.yudkowsky.net/virtues/ -> http://www.yudkowsky.net/rational/virtues/

Remember how in another post you argued a rationalist should be able to reserve his knowledge of it was taken away ? I believe this is a similar approach as the one taken by these hypothetical Jesuits. In fact, I see two possible ways to explain such a behavior : one could ask a physics student whether Newtonian physics were not the absolute best if they expected the student to discover relativity by themselves. Likewise, I guess the hypothetical Jesuits could want two separate benefits out of this : -Ensuring the student is savant/fanatic enough to join the tribe. -Teaching the student to discover core beliefs of their faith by themselves, both reinforcing these beliefs and assuring their correctedness.

So I grew up around Jesuits and, while I obviously can't speak for all of them, I'd say that they probably qualify as proto-rationalists, if not rationalists.  To the point where a large portion of other Christian sects denounce them as atheists because they refuse to wallow in mysticism like everyone else.

A core principle of the Jesuit philosophy is that God gave us our intellect specifically so that we could come to better understand him.  You won't find them trying to quibble about "micro" vs "macro" evolution or any of the other silliness that other groups use as a membership badge and try to talk in circles around.  They do still believe that there is a super-natural world beyond our ability to directly observe, but everything about this world must be logically consistent and any apparent inconsistency is a flaw in your own understanding, not a flaw in the world or a "divine mystery".

They are trained to draw a hard line between what they believe and what they know, and to treat any perceived inconsistency between the two as a reason to probe deeper until it makes sense.  And any fellow Christian who gives the appearance of engaging in "belief in belief"?  They'll tear him a new one just as fast as Yudkowsky would, if not faster.  They have his lack of tolerance for it, coupled with encyclopedic knowledge not only of the Bible's contents, but also generally of practically every work by every significant Christian and major pagan philosopher before or since.  

I suppose a good way to explain the fundamental difference is that where most Christian sects believe that certain things are true because they are in the Bible, the Jesuits would say that the stories in the Bible were selected because they teach a fundamental truth or two.  Were it not for the weight of Catholic tradition, I strongly suspect many Jesuits would be in favor of continuing to add to the anthology that is the Bible as we develop better stories for teaching the desired lessons.  Or, at least, developing an updated one that would make sense to a modern reader without having to spend decades studying all the cultural context necessary to understand what's going on.  I first heard the observation that "The Lord of the Rings is a fundamentally Christian story and worldview, just dressed up in different mythology" from a Jesuit for example.

Definitely interesting people and nearly always worth developing a relationship with when you can.  And while they'll try to convert you, they'll do it by presenting logical arguments, not by shouting and hitting you with a large book.  They'll take what they consider to be the core lessons and principles of Christianity and recompute how to explain them couched in your own world view.  And if you end up agreeing on everything but the mythology?  Well that's good enough.

I’m instantly thinking about politics: there are many cases where you cannot build a clear model of what’s going on exactly in a given government, and the information is not only not transparent but also degraded with all the noise, intentional and not. I think it’s reasonable to maintain the feeling of doubt while ruminating on such topics.