Annoyance comments on Cascio in The Atlantic, more on cognitive enhancement as existential risk mitigation - Less Wrong

20 [deleted] 18 June 2009 03:09PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (77)

You are viewing a single comment's thread. Show more comments above.

Comment author: Annoyance 22 June 2009 03:49:39PM 3 points [-]

I did not actively look for contradictory evidence.

I hate to discourage you when you're otherwise doing quite well, but the above is a major, major error.

Due to the human tendency towards confirmation bias, it's vastly important that you try to get a sense of the totality of the evidence, with a heavy emphasis on the evidence that contradicts your beliefs. If you have to prioritize, look for the contradicting stuff first.

Comment deleted 22 June 2009 04:13:51PM [-]
Comment author: Annoyance 22 June 2009 09:08:47PM -2 points [-]

I'd start getting very cautious and go make damn sure I wasn't wrong.

You should be doing that anyway.

But as the situation is ... I am not particularly incentivized to do this

Interesting. Does it bother you that you are not strongly motivated to avoid error?

Comment author: Alicorn 22 June 2009 09:14:31PM *  7 points [-]

There is a legitimate question of what errors are worth the time to avoid. Roko made a perfectly sensible statement - that it's not his top priority right now to develop immense certitude about this proposition, but it would become a higher priority if the answer became more important. It is entirely possible to spend all of one's time attempting to avoid error (less time necessary to eat etc. to remain alive and eradicate more error in the long run); I notice that you choose to spend a fair amount of your time making smart remarks to others here instead of doing that. Does it bother you that you are at certain times motivated to do things other than avoid some possible instances of error?

Comment author: Annoyance 22 June 2009 10:10:48PM -2 points [-]

Positive errors can be avoided by the simple expedient of not committing them. That usually carries very little cost.

Comment author: Alicorn 22 June 2009 10:23:13PM 0 points [-]

I agree completely, but this doesn't seem to be Roko's situation: he's simply not performing the positive action of seeking out certain evidence.

Comment author: Annoyance 23 June 2009 04:31:45PM -2 points [-]

But that action is a necessary part of producing a conclusion.

Holding a belief, without first going through the stages of searching for relevant data, is a positive error - one that can be avoided by the simple expedient of not reaching a conclusion before an evaluation process is complete. That costs nothing.

Asserting a conclusion is costly, in more than one way.

Comment author: thomblake 23 June 2009 04:45:45PM 1 point [-]

Humans hold beliefs about all sorts of things based on little or no thought at all. It can't really be avoided. It might be an open question whether one should do something about unjustified beliefs one notices one holds. And I don't think there's anything inherently wrong with asserting an unjustified belief.

Of course, I'm even using 'unjustified' above tentatively - it would be better to say "insufficiently justified for the context" in which case the problem goes away - certainly seeing what looks like a flower is sufficient justification for the belief that there is a flower, if nothing turns on it.

Not sure which sort of case Roko's is, though.

Comment author: Vladimir_Nesov 23 June 2009 05:50:38PM *  0 points [-]

At each point, you may reach a conclusion with some uncertainty. You expect the conclusion (certainty) to change as you learn more. It would be an error to immediately jump to inadequate levels of certainty, but not to pronounce an uncertain conclusion.