Sam_Jaques comments on How to Fix Science - Less Wrong

50 Post author: lukeprog 07 March 2012 02:51AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (141)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eugine_Nier 04 March 2012 01:00:24AM 5 points [-]

I thought the usual proposals/methods involved principally reporting log odds, to avoid exactly the issue of people having varying priors and updating on trials to get varying posteriors.

This only works in extremely simple cases.

Comment author: Sam_Jaques 05 March 2012 04:00:27PM 1 point [-]

Could you give an example of an experiment that would be too complex for log odds to be useful?

Comment author: Eugine_Nier 06 March 2012 02:25:38AM *  3 points [-]

Any example where there are more than two potential hypotheses.

Note, that for example, "this coin is unbiased", "this coin is biased toward heads with p=.61", and "this coin is biased toward heads with p=.62" count as three different hypotheses for this purpose.

Comment author: Cyan 10 March 2012 03:33:05AM 2 points [-]

This is fair as a criticism of log-odds, but in the example you give, one could avoid the issue of people having varying priors by just reporting the value of the likelihood function. However, this likelihood function reporting idea fails to be a practical summary in the context of massive models with lots of nuisance parameters.