= 27d567bc2d48d9f40e9ccc20ea370b15)
byrnema comments on Andrew Gelman on "the rhetorical power of anecdotes" - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (12)
Devil's advocate here: fictional 'evidence' can play the same role as a hypothetical; it creates a situation in which you can apply your intuition. If I encounter a story (e.g., an anecdote), it gets weighted to the extent that the story makes sense. If it doesn't make sense, it doesn't get weighted because I'll assume unknown factors or assign a low probability that it can be generalized. It doesn't matter so much, in either case, if the story is true or not.
Claiming that a story is true when it isn't is requesting more serious consideration of the plausibility of the scenario than it deserves, but if it the story is possible, it might as well have happened, since everything can happen once.
For example, with respect to the map anecdote, a person can use their own critical faculty (and experience) to decide the role that luck would have had in the survival of the soldiers. The anecdote certainly expressed the idea that impressions and attitude matter, but I also know that a map of the Starbucks locations in Nashville won't be so helpful in finding the Starbucks in Memphis. (...Except that there often will be one off any interstate exit near the downtown. So I might be lucky if I'm on the interstate and both cities have "Main" streets with a Starbucks. And it helps in any case to drive around.)
And there's another example, because I can't say I've actually ever seen a Starbucks on a street called Main Street, but the entire scenario seems plausible and illustrates the point.
I'm not sure I fully agree. Sure, if I can simulate draws from my posterior beliefs about reality, then I can reason from what those draws tell me. Scientists do this all the time when they simulate from a posterior distribution, and we hardly call this "fictional evidence" although, in a weak sense, it is. If that is all you are claiming, then I agree with you.
But when we create narratives to account for the evidence we see, we're almost never doing so by strictly drawing from a well-confirmed posterior. We're almost always doing whatever is simplest and whatever gratifies our immediate urges for availability and confirmation. In this sense, how can you really trust the narratives you generate in fiction? Sure, they might seem plausible to you, but how do you know? Have you really gone and made a probability calculation describing the whole chain of propositions necessary for your fictional narrative to be true? Almost surely not.
Therein lies great danger when you say something like: "... but if it the story is possible, it might as well have happened, since everything can happen once." I suggest that your Starbucks/Main Street example is a bad one, since these are rather specific details over which a given person's daily experience is likely to produce an accurate posterior distribution. Most instances of narrative fallacy are not this simple, and it would be a little disingenuous to claim that examples like that somehow lend validity to the entire practice of generalizing from fictional evidence.
More to the point, you should consider the LW post The Logical Fallacy of Generalizing from Fictional Evidence.
And in particular, in re-reading that, I noticed that Eliezer had hit upon Andrew Gelman's point as well:
There's a confusion regarding the example, due to my writing, because I meant to argue that the map of Nashville would not be useful for navigating Memphis. My thesis (however buried) was that a person can use anecdotes (fabricated or not) to evaluate how compelling an idea is. By analogy with the locations of Starbucks in different cities, I don't buy the idea that faith in a map is more important than the information content of the map, even if it somehow played a role in lost soldiers navigating their way out of the mountains.
I nearly always counter-weight my thoughts with counter-arguments, which is the way my brain organizes information, but which makes my writing difficult to follow, I'll work on that. In the original comment of mine above, I spent some time on the idea that to some extent the information of a map is relevant in a distinct but similar context, as for example in my analogy cities have spatial patterns in common (and mountains will too). But that was just a distracting counterpoint...
So in the end I think we agree mostly. My thesis was that a person needs to be critical of the relevance of anecdotes.
Where we might disagree is in the significance of the size of the domain in real life where anecdotes are the best means we have of organizing, extracting and relaying information. For example,
is going to be more or less useless in the cases where we are most dependent on narratives. Narratives help us integrate thinking over a non-linear network of ideas developed over a lifetime of experience. If estimating probabilities over a linear chain of propositions is feasible, then its a different kind of problem, one more suited to analytic analysis.
Back to the object level, what was the problem/idea the authors were trying to express with their story about the soldiers? That 'perspective and attitude' matters (more than? sometimes just as much as? can compensate for lack of?) real knowledge about the territory. It's a pretty amorphic, fuzzy idea to begin with. I consider it a success they were able to capture the idea at all, but I wouldn't consider it worth actually quantifying..