Cyan comments on Case study: abuse of frequentist statistics - Less Wrong

25 Post author: Cyan 21 February 2010 06:35AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (96)

You are viewing a single comment's thread. Show more comments above.

Comment author: wedrifid 23 February 2010 01:49:55AM *  2 points [-]

If you choose a single model to work with, you are effectively putting zero probability on all other models (that are not contained in your chosen model as sub-models).

I follow this reasoning and it applies in many cases. The reason I do not consider it applicable to the example given is due to the explicit mentioning of "We can make Z's pre-evidence probability arbitrarily small, to make this seem reasonable at the time." That changes the meaning of the example significantly in my understanding.

I claim that if Z is given enough consideration that 'arbitrarily small' is plugged in rather than mere exclusion from a model then it is just an error not an approximation. There are valid examples of bayes-in-practice that support the position John takes but I just don't consider this example a fair representation. Partly because the mistake is a bad way to handle urns and partly because explicitly plugging in bad priors for Z should make you explicitly expect bad posteriors for Z. Exclusion from the model itself is a different problem.

Comment author: Cyan 23 February 2010 02:14:50AM 1 point [-]

Good answer. I neglected to read up-thread with enough thoroughness.