You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Douglas_Knight comments on Open thread, 24-30 March 2014 - Less Wrong Discussion

6 Post author: Metus 25 March 2014 07:42AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (156)
Tags: open_thread

Comments (156)

You are viewing a single comment's thread. Show more comments above.

Comment author: IlyaShpitser 31 March 2014 11:57:49AM *  1 point [-]

So what I'm wondering is whether under frequentism P(hypothesis | data) is actually meaningless.

It's not meaningless, but people who follow R. A. Fisher's ideas for rejecting the null do not use p(hypothesis | data). "Meaningless" would be if frequentists literally did not have p(hypothesis | data) in their language, which is not true because they use probability theory just like everybody else.

Don't ask lesswrong about what frequentists claim, ask frequentists. Very few people on lesswrong are statisticians.

Comment author: Douglas_Knight 31 March 2014 07:41:11PM 0 points [-]

"Meaningless" would be if frequentists literally did not have p(hypothesis | data) in their language, which is not true because they use probability theory just like everybody else.

Many frequentists do insist that P(hypothesis) are meaningless, despite "using probability theory."

Comment author: IlyaShpitser 31 March 2014 08:13:34PM *  0 points [-]

Could you give me something to read? Who are these frequentists, and where do they insist on this?

Comment author: Douglas_Knight 31 March 2014 08:42:49PM 0 points [-]

Let us take a common phrase from the original comment "the hypothesis is either true or false". The first google hit:

There are two misconceptions that you must be aware of, as you will certainly hear these. The first is thinking that we calculate the probability of the null hypothesis being true or false. Whether the null hypothesis is true or false is not subject to chance; it either is true or it is false - there is no probability of one or the other.

Comment author: IlyaShpitser 31 March 2014 11:58:23PM *  0 points [-]

So from this statement you conclude that frequentists think P(hypothesis) is meaningless? Bayesians assign degrees of belief to things that are actually true or false also. The coin really is either fair or not fair, but you will never find out with finite trials. This is a map/territory distinction, I am surprised you didn't get it. This quote has nothing to do with B/F differences.

A Bayesian version of this quote would point out that it is a type error to confuse the truth value of the underlying thing, and the belief about this truth value.

Comment author: Oscar_Cunningham 01 April 2014 09:02:25AM 0 points [-]

You have successfully explained why it is irrational for frequentists to consider P(hypothesis) meaningless. And yet they do. They would say that probabilities can only be defined as limiting frequencies in repeated experiments, and that for a typical hypothesis there is no experiment you can rerun to get a sample for the truth of the hypothesis.

Comment author: IlyaShpitser 01 April 2014 09:22:49AM *  0 points [-]

You guys need to stop assuming frequentists are morons. Here are posts by a frequentist:

http://normaldeviate.wordpress.com/2012/12/04/nate-silver-is-a-frequentist-review-of-the-signal-and-the-noise/

http://normaldeviate.wordpress.com/2012/11/17/what-is-bayesianfrequentist-inference/

Some of the comments are good as well.

Comment author: Oscar_Cunningham 01 April 2014 11:15:21AM *  2 points [-]

Yes, you're right. Clearly many people who identify as frequentists do hold P(hypothesis) to be meaningful. There are statisticians all over the B/F spectrum as well as not on the spectrum at all. So when I said "frequentists believe ..." I could never really be correct because various frequentists believe various different things.

Perhaps we could agree on the following statement: "Probabilities such as P(hypothesis) are never needed to do frequentist analysis."

For example, the link you gave suggests the following as a characterisation of frequentism:

Goal of Frequentist Inference: Construct procedure with frequency guarantees. (For example, confidence intervals.)

Since frequency guarantees are typically of the form "for each possible true value of theta doing the construction blah on the data will, with probability at least 1-p, yield a result with property blah". Then since this must hold true for each theta, the distribution for the true value of theta is irrelevant.

Comment author: IlyaShpitser 01 April 2014 12:15:59PM *  0 points [-]

I could never really be correct because various frequentists believe various different things.

The interesting questions to me are: (a) "what is the steelman of the frequentist position?" (folks like Larry are useful here), and (b) "are there actually prominent frequentist statisticians who say stupid things?"

By (b) I mean "actually stupid under any reasonable interpretation."

Clearly many people who identify as frequentists

Quote from the url I linked:

One thing that has harmed statistics — and harmed science — is identity statistics. By this I mean that some people identify themselves as “Bayesians” or “Frequentists.” Once you attach a label to yourself, you have painted yourself in a corner.

When I was a student, I took a seminar course from Art Dempster. He was the one who suggested to me that it was silly to describe a person as being Bayesian of Frequentist. Instead, he suggested that we describe a particular data analysis as being Bayesian of Frequentist. But we shouldn’t label a person that way.

I think Art’s advice was very wise.

"Keep your identity small" -- advice familiar to a LW audience.

Perhaps we could agree on the following statement: "Probabilities such as P(hypothesis) are never needed to do frequentist analysis."

I guess you disagree with Larry's take: B vs F is about goals not methods. I could do Bayesian looking things while having a frequentist interpretation in mind.

In the spirit of collaborative argumentation, can we agree on the following:

We have better things to do than engage in identity politics.

Powered by Reddit Powered by Reddit