David_Gerard comments on The failure of counter-arguments argument - Less Wrong

14 Post author: Stuart_Armstrong 10 July 2013 01:38PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (43)

You are viewing a single comment's thread. Show more comments above.

Comment author: AlexMennen 10 July 2013 04:09:19PM 3 points [-]

I agree that the counterarguments to AI risk seem quite poor (to me), and this is some evidence in favor of AI risk. However, since humans are biased towards not being convinced by any counterargument to something they strongly believe, this limits how much we can use the unconvincingness of counterarguments to AI risk to update in favor of AI risk.

Comment author: David_Gerard 10 July 2013 07:44:59PM *  0 points [-]

However, since humans are biased towards not being convinced by any counterargument to something they strongly believe

Indeed. It has been experimentally tested that strong counterarguments are less likely to convince a strong believer.

“The general idea is that it’s absolutely threatening to admit you’re wrong,” says political scientist Brendan Nyhan, the lead researcher on the Michigan study. The phenomenon — known as “backfire” — is “a natural defense mechanism to avoid that cognitive dissonance.”

(I'm sure this was linked on LW before, but couldn't find it quickly ...)

So the proposed "failure of counter-arguments argument" falls afoul of errors in human thinking.

Comment author: gwern 10 July 2013 08:22:44PM *  15 points [-]

The backfire effect, as far as I know, has never been replicated and bears all the hallmarks of being your classic counterintuitive psych finding which will disappear the moment anyone looks too hard at it. I've been trying to discourage people from citing it...

Comment author: gjm 13 July 2013 01:59:46PM 3 points [-]

I've been trying to discourage people from citing it...

Nononono, don't do that! It'll just make them believe it more strongly.

(The nice thing is that actually your attempts at discouragement will, roughly speaking, work if and only if the discouragement is correct!)

Comment author: ThisSpaceAvailable 15 July 2013 02:54:26AM 1 point [-]

Or maybe the evidence for the effect is so strong that you refuse to believe it. :)

Comment author: gwern 15 July 2013 03:39:46AM 1 point [-]

I feel obligated to point out that the backfire effect, even in the original paper, applied only to a few zealots, and not everyone and not relatively moderate subjects. So your obvious joke is not itself consistent with the paper.

Comment author: Jayson_Virissimo 13 July 2013 01:09:48AM 1 point [-]

What other (popular) psychology findings do you think won't hold up?

Comment author: gwern 13 July 2013 01:27:44AM 3 points [-]

Stereotype threat looks extremely questionable to me, and dual n-back is more or less finished as far as I'm concerned. Those are the only two famous findings which I can think of off-hand; did you have any specific ones in mind?