loqi comments on Beautiful Probability - Less Wrong

34 Post author: Eliezer_Yudkowsky 14 January 2008 07:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (109)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: LauWren 16 February 2011 02:48:41PM *  1 point [-]

We have additional information about researcher 2's experiment. If researcher 2 didn't look at the data before that point, then the procedures were the same, so the data should be treated the same.

If researcher 2 did check the data along the way--a reasonable enough assumption, given researcher 2's goal--then there were other tests which all came out below 60%. There was an upswing in successes at the end, and we know it. The other experiment may well have experienced the same thing, but in experiment 2, I don't have to look; I see it. Was there an important variable that we overlooked?

Of course, this is only due to extra information I happen to have about the latter. I haven't bothered to check whether something similar was there for the previous, because there wasn't anything to make it catch my attention. If I have the tools to do so, I would still like to treat them both the same--I want to see researcher 1's results lined up by time as well. If the upswing is repeated...well, that's funny.

Comment author: loqi 12 October 2011 05:24:05AM 0 points [-]

Good point, there is some ordering information leaked. This is consistent with identical likelihoods for both setups - learning which permutation of arguments we're feeding into a commutative operator (multiplication of likelihood ratios) doesn't tell us anything about its result.