Andrew Gelman has a post up today discussing a particularly illustrative instance of narrative fallacy involving the recent plagiarism discussion surrounding Karl Weick. I think there are also some interesting lessons in there about generalizing from fictional evidence.

In particular, Gelman says, "Setting aside [any] issues of plagiarism and rulebreaking, I argue that by hiding the source of the story and changing its form, Weick and his management-science audience are losing their ability to get anything out of it beyond empty confirmation."

I am wondering if anyone has explicitly looked into connections between generalizing from fictional evidence and confirmation bias. It sounds intuitively plausible that if you are going to manipulate fictional evidence for your purposes, you'll almost always come out believing the evidence has confirmed your existing beliefs. I would be highly interested in documented accounts where the opposite has happened and fictional evidence actually served as a correction factor.

For what it's worth, I personally enjoy a watered-down version of the moral that Weick attempts to manipulate from the story that's discussed in Gelman's post. My high school math teacher used to always say to us, "When you don't know what to do, do something." I think he said it because he was constantly pissed about questions left completely blank on his math exams, and wanted students to write down scribblings or ideas so he could at least give them some partial credit, but it has been more motivational than that for me.