ChrisHallquist comments on The Statistician's Fallacy - Less Wrong

38 Post author: ChrisHallquist 09 December 2013 04:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (67)

CC Licenced

Comments (67)

Sort By: Best

You are viewing a single comment's thread. Show more comments above.

Comment author: IlyaShpitser 11 December 2013 04:37:28PM * 20 points [-]

The professor had a background in statistics, and as far as I could tell knew her stuff in that area (though she dismissed Bayesianism in favor of orthodox statistics).

Bayesians will realize that, since there's a good chance that of happening even when the conclusion is correct and well- supported by the evidence, finding mistakes in the statistics is only weak evidence that the conclusion is wrong.

Wow, lesswrong, you just never fail to do this at every opportunity. Bayesianity is not a minority view anymore. Bayesians do not have a monopoly on correct reasoning with probabilities. Seriously, knock it off, please.

The professor had a background in statistics

Do you have a background in statistics, Chris?

edit: One of the areas I am working on is "causal discovery," which is learning the structure of graphs from observational data. One problem I have worked on a lot is causal discovery in the presence of hidden variables. It turns out there is a very interesting statistical model that recovers all independence constraints that a hidden variable DAG imposes on the observed margin. It also turns out that there is a way to write down the likelihood for this model in the case of discrete state spaces, while doing the same for continuous state spaces is currently unknown. This suggests that a search and score method (e.g. Bayesian method, or at least a method with a Bayesian justification) is natural for the discrete case, while a method based on hypothesis testing (e.g. a frequentist method, although Bayesian versions are possible here, they are less satisfactory because there is no global posterior) is natural for the continuous case. After all, we can't very well figure out what the posterior is if we can't even write the likelihood down.

Did the above paragraph make sense to you? These are the kinds of consideration people have in mind when thinking about B vs F. If you aren't working in ML/stats I am not sure what the point even is of having an opinion on this topic, other than "belief as attire."

It's completely bizarre. Somehow when it comes to B vs F, LW is willing to tell experts what they should be doing in their area of expertise.

Comment author: ChrisHallquist comment score below threshold [+] (9 children)

Comment author: Vaniver 11 December 2013 10:13:14PM * 5 points [-]

Oh my god! You're right! How dare people on LessWrong mention their support for a view that's been argued for at great length on here! The horror! The horror!

I'm ambivalent about this, actually. I think that interpreting probability as subjective uncertainty is more sensible, I think the Bayesian toolkit is superior for a wide array of problems, and I think that any technique generally called 'Frequentist' probably has an equivalent Bayesian interpretation.

But I generally get the sense that the Bayesian-Frequentist divide is unproductive, and I am disappointed when LW seems to widen that divide.

Unless you're an active statistician, I don't think you should expect you have a solid view of what statisticians currently think. Consider this other comment, where JoshuaZ points out that, at this point, Bayesianism is part of 'orthodox statistics,' and if you meant to say she was a Frequentist you should have said that directly.

Comment author: [deleted] 12 December 2013 12:27:30AM 4 points [-]

But I generally get the sense that the Bayesian-Frequentist divide is unproductive, and I am disappointed when LW seems to widen that divide.

Exactly; I couldn't have put it better myself (and indeed, didn't).

Comment author: [deleted] 11 December 2013 10:00:02PM 3 points [-]

What your professor actually said about Bayesianism is irrelevant; GP is responding to what you said about your professor.

Even before the edit, Ilya had a valid point; after the edit, you look like someone whose identity as a Bayesian has gotten in the way of thinking. No matter what you do, cut toward your enemy.

Comment author: ChrisHallquist 11 December 2013 11:59:35PM -1 points [-]

I replaced "orthodox statistics" with "frequentism" in the post in case that will make people happy, but as I understood him Ilya, wasn't just complaining about that, but also my own implied support for Bayesianism over frequentism. And maybe the standard LessWrong position on that debate is wrong, but to come in and announce that the LW view is wrong without argument, when it's been argued for at such great length, seems odd to put it midly.

Ilya comes across as not being aware of how much Eliezer and other people here have written about that debate. In fact, it's not even clear to me if he understands what someone like Eliezer (or for that matter, an academic epistemologist) means when they say "Bayesianism."

Comment author: Mayo 13 December 2013 03:17:29AM 4 points [-]

I realize Eliezer holds great sway on this blog, but I think people here ought to question a bit more closely some of his most winning arguments in favor of casting out frequents for Bayesianism. I've only read this blog around 4 times, and each time I've found a howler apparently accepted. But putting those aside, I find it curious that the results on psychological biases that is given so much weight on this blog are arrived at and affirmed by means of error statistical methodology. error statistics.com

Comment author: gwern 13 December 2013 04:45:42AM * 4 points [-]

But putting those aside, I find it curious that the results on psychological biases that is given so much weight on this blog are arrived at and affirmed by means of error statistical methodology.

Speaking as one of the LWers who has spent a fair bit of time reading up on both the heuristics & biases literature and also the problems & misuse of NHST (although I certainly couldn't compare to your general statistical expertise), my position is basically that there's no available literature which have examined the H&B topic with a superior methodology (so there's no alternative we could use) and that on the whole H&B has found real effects despite the serious weaknesses in the methodology - for example, of the Reproducibility Project's 13 targets, the ones which failed to replicate were priming effects and not the tested H&B effects (eg. sunk costs, anchoring, framing). The problems are not so bad as to drain the H&B results of all validity, just some.

So while the H&B research program is no doubt undermined and hampered by the statistical tools and practices of the researchers involved, there seem little reason to think that the most-discussed biases are pure statistical mirages; and so they are entirely relevant to our discussions here.

(From my perspective, the real question about the utility of the H&B literature to our practical discussions here on LW is not whether they exist in the lab settings they are studied in - it's clear that they are not artifacts of p-value hacking or anything like that - but whether they operate in the real world to a meaningful extent and shape opinions & actions on a wide scale and on the topics we care about. This is, unfortunately, something which is very difficult to study no matter what methodology one might choose to use, and for this concern, criticizing the use of error statistical methodology is largely irrelevant.)

Comment author: [deleted] 12 December 2013 12:26:20AM * 1 point [-]

[They were also complaining about] my own implied support for Bayesianism over frequentism.

I don't see that anywhere. It's clear the majority of LessWrong (the actual subject of Ilya's actual sentences) thinks Bayesian statistics (who was talking about epistemology?) is better -- with the possible exception of gwern, who uses whatever is most pragmatic (and who I personally think is the actual winner of this debate).

Ilya comes across as not being aware of how much Eliezer and other people here have written about that debate. In fact, it's not even clear to me if he understands what someone like Eliezer (or for that matter, an academic epistemologist) means when they say "Bayesianism."

Even a casual examination of their comment record (or, alternatively, a Google search) would have demonstrated that you're completely wrong in your assessment. I don't know any other regular on the site that knows more about statistics.

Comment author: Cyan 13 December 2013 04:40:50AM 3 points [-]

I don't know any other regular on the site that knows more about statistics.

Allow me to introduce myself. I make my living as a biostatistician; I am philosophically a Jaynesian, in practice a statistical ecumenist. (I don't know and don't claim to know causal inference in anything like the depth that Ilya does -- that's his specialty, just like mine is Bayesian modeling.)

Comment author: [deleted] 13 December 2013 06:35:57PM 1 point [-]

Nice to meet ya! Consider myself updated :)

ChrisHallquist comments on The Statistician's Fallacy - Less Wrong

Article Navigation

Comments (67)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (67)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login