cousin_it comments on Open Thread June 2010, Part 4 - Less Wrong

5 Post author: Will_Newsome 19 June 2010 04:34AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (325)

You are viewing a single comment's thread. Show more comments above.

Comment author: cousin_it 01 July 2010 10:36:24AM *  1 point [-]

(A more "Bayesian" alternative to model checking is to have an explicit "none of the above" hypothesis as part of your prior.)

I don't see how that's possible. How do you compute the likelihood of the NOTA hypothesis given the data?

Comment author: Cyan 02 July 2010 03:04:36PM *  2 points [-]

NOTA is not well-specified in the general case, but in at least one specific case it's been done. Jaynes's student Larry Bretthorst made a useable NOTA hypothesis in a simplified version of a radar target identification problem (link to a pdf of the doc).

(Somewhat bizarrely, the same sort of approach could probably be made to work in certain problems in proteomics in which the data-generating process shares the key features of the data-generating process in Bretthorst's simplified problem.)

Comment author: cousin_it 02 July 2010 04:30:49PM *  0 points [-]

If I'm not mistaken, such problems would contain some enumerated hypotheses - point peaks in a well-defined parameter space - and the NOTA hypothesis would be a uniformly thin layer over the rest of that space. Can't tell what key features the data-generating process must have, though. Or am I failing reading comprehension again?

Comment author: Cyan 02 July 2010 08:24:57PM *  0 points [-]

If I'm not mistaken, such problems would contain some enumerated hypotheses - point peaks in a well-defined parameter space - and the NOTA hypothesis would be a uniformly thin layer over the rest of that space

Yep.

Can't tell what key features the data-generating process must have, though.

I think the key features that make the NOTA hypothesis feasible are (i) all possible hypotheses generate signals of a known form (but with free parameters), and (ii) although the space of all possible hypotheses is too large to enumerate, we have a partial library of "interesting" hypotheses of particularly high prior probability for which the generated signals are known even more specifically than in the general case.