EHeller comments on Don't You Care If It Works? - Part 1 - Less Wrong

4 Post author: Jacobian 29 July 2015 02:32PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (58)

Comments (58)

You are viewing a single comment's thread. Show more comments above.

Comment author: EHeller 30 July 2015 03:16:45AM *  0 points [-]

No, there's a limit on that as well. See http://www.ejwagenmakers.com/2007/StoppingRuleAppendix.pdf

I can check my simulation for bugs. I don't have the referenced textbook to check the result being suggested.

It is my thesis that every optional stopping so-called paradox can be converted into a form without optional stopping, and those will be clearer as to whether the problem is real or not.

The first part of this is trivially true. Replace the original distribution with the sampling distribution from the stopped problem, and it's not longer a stopped problem, it's normal pulls from that sampling distribution.

I'm not sure it's more clear,I think it is not. Your "remapped" problem makes it look like it's a result of low-data-volume and not a problem of how the sampling distribution was actually constructed.

Comment author: ike 30 July 2015 03:27:53AM 0 points [-]

I don't have the referenced textbook to check the result being suggested.

You can see http://projecteuclid.org/euclid.aoms/1177704038, which proves the result.

Replace the original distribution with the sampling distribution from the stopped problem, and it's not longer a stopped problem, it's normal pulls from that sampling distribution.

How would this affect a frequentist?

I'm not sure it's more clear,I think it is not. Your "remapped" problem makes it look like it's a result of low-data-volume and not a problem of how the sampling distribution was actually constructed.

I'm giving low data because those are the simplest kinds of cases to think of. If you had lots of data with the same distribution/likelihood, it would be the same. I leave it as an exercise to find a case with lots of data and the same underlying distribution ...

I was mainly trying to convince you that nothing's actually wrong with having 33% false positive rate in contrived cases.

Comment author: EHeller 30 July 2015 04:51:38AM 0 points [-]

How would this affect a frequentist?

It doesn't the frequentist is already measuring with the sample distribution. That is how frequentism works.

I was mainly trying to convince you that nothing's actually wrong with having 33% false positive rate in contrived cases.

I mean it's not "wrong" but if you care about false positive rates and there is a method had has a 5% false positive rate, wouldn't you want to use that instead?

Comment author: ike 30 July 2015 05:46:33AM 0 points [-]

If for some reason low false positive rates were important, sure. If it's important enough to give up consistency.

Powered by Reddit Powered by Reddit