Comment on The Importance of Self-Doubt

Fri, 20 Aug 2010 11:55:52 GMT

I don't object to devoting (almost) all efforts to a single cause generally. I do, however, object to such devotion in case of FAI and the Singularity.

If a person devotes all his efforts to a single cause, his subjective feeling of importance of the cause will probably increase and most people will subsequently overestimate how important the cause is. This danger can be faced by carefully comparing the results of one's deeds with the results of other people's efforts, using a set of selected objective criteria, or measure it using some scale ideally fixed at the beginning, to protect oneself from moving the goalposts.

The problem is, if the cause is put so far in the future and based so much on speculations, there is no fixed point to look at when countering one's own biases, and the risk of a gross overestimation of one's agenda becomes huge. So the reason why I dislike the mentioned suggestions (and I am speaking, for example, about the idea that it is a strict moral duty for everybody who can to support the FAI research as much as they can, which were implicitly present at least in the discussions about the forbidden topic) is not that I reject single-cause devotion in principle (although I like to be wary about it in most situations), but that I assign too low probability to the correctness of the underlying ideas. The whole business is based on future predictions of several tens or possibly hunderts years in advance, which is historically a very unsuccessful discipline. And I can't help but include it in that reference class.

Simultaneously, I don't accept the argument of very huge utility difference between possible outcomes, which should justify one's involvement even if the probability of success (or even probability that the effort has sense) is extremely low. Pascal-wageresque reasoning is unreliable, even if formalised, because it needs careful and precise estimation of probabilities close to 1 or 0, which humans are provably bad at.

Discuss