Yvain comments on The Statistician's Fallacy - Less Wrong

38 Post author: ChrisHallquist 09 December 2013 04:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (67)

You are viewing a single comment's thread.

Comment author: Yvain 09 December 2013 07:30:27PM 20 points [-]

Thank you for taking our ability to dismiss experts we don't like up one meta-level.

Comment author: Cyan 10 December 2013 05:02:42PM *  3 points [-]

The message is really that the impact of a mistake has to be assessed in light of the entire body of evidence supporting the experts' view. Sometimes a single mistake really does undermine a case, but mostly not.

Comment author: ChrisHallquist 09 December 2013 08:57:38PM *  3 points [-]

I don't think the heuristic I'm advocating is terribly prone to abuse:

  • If a statistician and an expert on [other field] disagree on [other field], I advocate tending to side with the expert on [other field], especially if there's a strong consensus among experts on [other field] on that point.
  • If couse when a statistician and an expert on [other field] disagree on statistics, I advocate trusting the statistician.

Maybe I should add tack this on to the end of the post?

Comment author: Lumifer 10 December 2013 05:05:37AM 6 points [-]

Your two points easily combine into one: when two experts in different fields disagree, you should trust the expert in whose field of expertise the point of contention is. That's not a particularly new piece of advice (and I personally am suspicious of its generality).

Comment author: ChrisHallquist 10 December 2013 05:16:04AM 3 points [-]

No, it's not particularly novel. The novel thing is warning against thinking that becasue stats is used in so many other disciplines, stats is an exception.

Comment author: Lumifer 10 December 2013 05:32:04AM 3 points [-]

Well, that does depend on "in whose field of expertise the point of contention is". If a professional statistician says that a particular study screwed up its statistical analysis, I have little reason (ceteris paribus) to disbelieve him. That, of course, doesn't mean the paper should go into the wastebasket, maybe it just needs a minor correction -- but it's very hard to generalize about such things.

Certainly, when statisticians start to talk about, say, biology, you should be suspicious -- but no more than you should be suspicious of climate scientists talking about economics.