Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

What Evidence Filtered Evidence?

43 Eliezer_Yudkowsky 29 September 2007 11:10PM

Yesterday I discussed the dilemma of the clever arguer, hired to sell you a box that may or may not contain a diamond.  The clever arguer points out to you that the box has a blue stamp, and it is a valid known fact that diamond-containing boxes are more likely than empty boxes to bear a blue stamp.  What happens at this point, from a Bayesian perspective?  Must you helplessly update your probabilities, as the clever arguer wishes?

If you can look at the box yourself, you can add up all the signs yourself.  What if you can't look?  What if the only evidence you have is the word of the clever arguer, who is legally constrained to make only true statements, but does not tell you everything he knows?  Each statement that he makes is valid evidence—how could you not update your probabilities?  Has it ceased to be true that, in such-and-such a proportion of Everett branches or Tegmark duplicates in which box B has a blue stamp, box B contains a diamond?  According to Jaynes, a Bayesian must always condition on all known evidence, on pain of paradox.  But then the clever arguer can make you believe anything he chooses, if there is a sufficient variety of signs to selectively report.  That doesn't sound right.

continue reading »