Vaniver comments on Evidence and counterexample to positive relevance - Less Wrong

-2 Post author: fsopho 25 May 2013 06:40PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (37)

You are viewing a single comment's thread. Show more comments above.

Comment author: elharo 26 May 2013 12:14:19PM *  0 points [-]

I think your example eviscerates the first "Increase in Probability" definition, at least as presented here, and shows that it doesn't account for non-independent evidence. If I wake up one morning and read in the New York Times reports that Bill Clinton has bought 99.9 % of tickets in a lottery, this strongly increases my estimate of the probability that Clinton will win the lottery. Reading the same story in the Washington Post does not materially increase my estimate given the background information in the New York Times. (I suppose it slightly reduces the probability that the Times story is a hoax or mistaken. Just maybe that's relevant.) Thus by this definition the Post story is not evidence (or at least is very weak evidence) that Bill Clinton will win the lottery.

However, suppose instead I wake up and read the Post story first. This now provides strong evidence that Bill Clinton will the lottery. The Times story is weak evidence at best. So depending on the irrelevant detail of which story I read first, one is strong evidence and one is weak evidence? That seems wrong. I don't want the strength of evidence to depend more on the irrelevant detail of which order I encounter two pieces of evidence. So perhaps what's being defined here is not the quality of the evidence but the usefulness of new evidence to me given what I already know?

Of course evidence and probabilities, especially non-independent probabilities are not additive.

This is not inobvious, so I notice that I am confused. I have to think that I'm misunderstanding this definition; or that there are details in the book that you're not reporting.

Comment author: Vaniver 26 May 2013 05:37:31PM *  2 points [-]

I think your example eviscerates the first "Increase in Probability" definition, at least as presented here, and shows that it doesn't account for non-independent evidence.

There's a deep philosophical point at stake here. Is probability a 1) quantification of a person's uncertainty, or 2) a statement about the universe?

2) is a position that is not well-regarded here, and I would recommend Probability is in the Mind and then possibly Probability is Subjectively Objective as to why.

If 1), then their previous knowledge matters. It's one particular person's uncertainty. You start off very uncertain about the lottery result. Then you read the first newspaper report, which makes you much less uncertain about the lottery result. Then you read the second newspaper report, which makes you very slightly less uncertain about the lottery result. As you watch your uncertainty decrease, you see that the first report has a huge effect (and thus is strong evidence), and the second report has a small effect (and thus is weak evidence). With more background knowledge about correctness, the second report drops to 0 effect (and thus is not evidence to you).

The mathematical way to think about this is that the strength of evidence is a function of both e and b. This is necessary to ensure that there's a probability shift, and if you don't force a shift, you can have even more silly 'evidence', like sock color being evidence for logical tautologies.

One might object that they're not particularly interested in measuring their personal uncertainty, but in affecting the beliefs of others. If you wanted to convince someone else that Bill Clinton is probably going to win the lottery, it seems reasonable to be indifferent to whether they read the Times or the Post, so long as they read one. But your personal measure of evidence is wildly different between the two papers! How do we reconcile your personal measure of uncertainty, and your desire to communicate effectively?

The answer I would give is being more explicit about the background b. It's part of our function, and so let's acknowledge it. When b is "b & e1", then e2 is not significant evidence. When b is just b, e2 is significant evidence, and so if you want to convince someone else of h and their current knowledge is just b, you can be indifferent between e1 and e2 because P(h|b,e1)=P(h|b,e2).

Let's further demonstrate that with a modification to the counterexample like the one Manfred suggested. Suppose I learn e1, that the New York Times reports that Bill Clinton owns all but one of the tickets, by reading that day's copy of the Times, which I'll call r1.

Suppose my background, which I'll call t, is that the New York Times (and my perception of it) is truthful. So P(e1|t,r1)=1. I liked the prose so much, I give the newspaper a second read, which I'll call r2. Is r2 evidence for e1? Well, P(e1|t,r1,r2)=1, which is the same as P(e1|t,r1), which suggests by the "increase in probability" definition that reading the newspaper a second time is not evidence for what the newspaper says. (Note that if we relax the assumption that my reading comprehension is perfect, then reading something a second time is evidence for what I thought it said, assuming I think the same thing after the second read. If we only relax the assumption that the newspapers are perfectly correct, we don't get a change in evidence.)

Does it seem reasonable that, given perfect reading comprehension, that considering the same piece of evidence twice should only move your uncertainty once? If so, what is the difference between that and the counterexample where one newspaper says "Times" on the front, and the other says "Post"?

(If you relax the assumption that the newspapers are perfectly correct, then the second newspaper is evidence by the "increase in probability" definition, because of the proposition g discussed in the OP.)

Comment author: fsopho 26 May 2013 10:19:04PM *  0 points [-]

Right, so, one think that is left open by both definitions is the kind of interpretation given to the function P. Is that suppose to be interpreted as a (rational) credence function? If so, the Positive Relevance account would say that e is evidence that h when one is rational in having a bigger credence in h when one has e as evidence than when one does not have e as evidence. For some, though, it would seem that in our case the agent that already knows b and e1 wouldn't be rational in having a bigger credence that Bill will win the lottery if she learns e2.

But I think we can try to solve the problem without having to deal with the interpretation of the probability issue. One way to go, for the defender of the Positive Relevance account, would be to say that the counterexample assumes a universal quantification over the conditionalizing sentence that was not intended - one would be interpreting Positive Relevance as saying:

  • (For every background b) e is evidence that h iff P(h|e&b) > P(h|b)

But such interpretation, the defender of Positive Relevance could say, is wrong, and it is wrong just because of the kinds of examples as the one presented in the post. So, in order for e2 to be evidence that h, e2 does not need to increase the probability of h conditional on every conceivable background b. Specifically, it doesn't need to increase the probability of h conditional on b when b contains e1, for example. But how would the definition look like without such quantification. Well, I don't quite know sufficiently about it yet (this is new to me), but I think that maybe the following would do:

  • (For every tautology b) e is evidence that h iff P(h|e&b) > P(h|b)

The new definition does not require e to increase h's probability conditional on every possible background. How does that sound?

Comment author: Vaniver 27 May 2013 02:27:26AM 1 point [-]

It's not clear to me why exactly you want the definition of evidence to not rely on the particular background of the mind where the P resides.

If you limit b to tautologies, you kill its usefulness. "This is a fair lottery in which one ticket drawn at random will win" isn't a tautology.

Comment author: fsopho 27 May 2013 02:16:17PM 0 points [-]

But that these are the truth conditions for evidential support relations does not mean that only tautologies can be evidence, nor that only sets of tautologies can be one's background. If you prefer, this is supposed to be a 'test' for checking if particular bits of information are evidence for something else. So I agree that backgrounds in minds is one of the things we got to be interested in, as long as we want to say something about rationality. I just don't think that the usefulness of the test (the new truth-conditions) is killed. =]