gwern comments on Rationality Quotes September 2011 - Less Wrong

7 Post author: dvasya 02 September 2011 07:38AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (482)

You are viewing a single comment's thread. Show more comments above.

Comment author: Will_Newsome 10 September 2011 01:39:33PM 2 points [-]

I think the literature disagrees. E.g. good players are less prone to confirmation bias and I think that this is transferable. (Google Scholar would know better.) Introspectively I feel like playing chess makes me a better thinker. Chess is memorization of moves and strategies only in the sense that guitar is memorization of scales and chords. You need them to play well but they're not sufficient.

Comment author: gwern 10 September 2011 08:49:39PM 5 points [-]

E.g. good players are less prone to confirmation bias

True; see 2004 "Chess Masters' Hypothesis Testing" Cowley & Bryne:

But experimental evidence from studies of reasoning shows that people often find falsification difficult. We suggest that domain expertise may facilitate falsification. We consider new experimental data about chess experts’ hypothesis testing. The results show that chess masters were readily able to falsify their plans. They generated move sequences that falsified their plans more readily than novice players, who tended to confirm their plans. The finding that experts in a domain are more likely to falsify their hypotheses has important implications for the debate about human rationality.

I think that this is transferable

Well... The chess literature and general literature on learning rarely finds transfer. From the Nature coverage of that study:

Byrne and Cowley now hope to study developing chess players to find out how and when they develop falsification strategies. They also want to test chess masters in other activities that involve testing hypotheses - such as logic problems - to discover if their falsification skill is transferable. On this point Orr is more sceptical: "I've never felt that chess skills cross over like that, it's a very specific skill."

Checking Google Scholar, I see only one apparent followup, the 2005 paper by the same authors, "When falsification is the only path to truth":

Can people consistently attempt to falsify, that is, search for refuting evidence, when testing the truth of hypotheses? Experimental evidence indicates that people tend to search for confirming evidence. We report two novel experiments that show that people can consistently falsify when it is the only helpful strategy. Experiment 1 showed that participants readily falsified somebody else’s hypothesis. Their task was to test a hypothesis belonging to an ‘imaginary participant’ and they knew it was a low quality hypothesis. Experiment 2 showed that participants were able to falsify a low quality hypothesis belonging to an imaginary participant more readily than their own low quality hypothesis. The results have important implications for theories of hypothesis testing and human rationality.

While interesting and very relevant to some things (like programmers' practice of 'rubber ducking' - explaining their problem to an imaginary creature), it doesn't directly address chess transfer.