I think we can combine your [cousin_it's] suggestion with MrMind's for an Option 2 scenario.
Suppose Bob finds that he has a stored belief in Bright with an apparent memory of having based it on evidence A, but no memory of what evidence A was. That does constitute some small evidence in favor of Bright existing.
But if Bob then goes out in search of evidence about whether Bright exists, and finds some evidence A in favor, he is unable to know whether it's the same evidence as before that he had forgotten, or if it's different evidence. Another way of saying that is that Bob can't tell whether or not A and A are independent. I suppose the ideal reasoner's response would be to assign a probability density distribution over a range from full independence to full dependence and proceed with any belief updates taking that distribution into account.
The distribution should be formed by consideration of how Bob got the evidence. If Bob found his new evidence A in some easily repeatable way, like hearing it from Bright apologists, then Bob would probably think dependence on A is much more likely than independence, and so he would take into account mostly just A and not A. But if Bob got A by some means that he probably wouldn't have had access to in the past, like an experiment requiring brand new technology to perform, then he would probably think independence was more likely, and so he would take into account A and A mostly separately.
This article is going to be in the form of a story, since I want to lay out all the premises in a clear way. There's a related question about religious belief.
Let's suppose that there's a country called Faerie. I have a book about this country which describes all people living there as rational individuals (in a traditional sense). Furthermore, it states that some people in Faerie believe that there may be some individuals there known as sorcerers. No one has ever seen one, but they may or may not interfere in people's lives in subtle ways. Sorcerers are believed to be such that there can't be more than one of them around and they can't act outside of Faerie. There are 4 common belief systems present in Faerie:
This is completely exhaustive, because everyone believes there can be at most one sorcerer. Of course, some individuals within each group have different ideas about what their sorcerer is like, but within each group they all absolutely agree with their dogma as stated above.
Since I don't believe in sorcery, a priori I assign very high probability for case 4, and very low (and equal) probability for the other 3.
I can't visit Faerie, but I am permitted to do a scientific phone poll. I call some random person, named Bob. It turns out he believes in Bright. Since P(Bob believes in Bright | case 1 is true) is higher than the unconditional probability, I believe I should adjust the probability of case 1 up, by Bayes rule. Does everyone agree? Likewise, the probability of case 3 should go up, since disbelief in Dark is evidence for existence of Dark in exactly the same way, although perhaps to a smaller degree. I also think the case 2 and case 4 have to lose some probability, since it adds up to 1. If I further call a second person, Daisy, who turns out to believe in Dark, I should adjust all probabilities in the opposite direction. I am not asking either of them about the actual evidence they have, just what they believe.
I think this is straightforward so far. Here's the confusing part. It turns out that both Bob and Daisy are themselves aware of this argument. So, Bob says, one of the reasons he believes in Bright, is because that's positive evidence for Bright's existence. And Daisy believes in Dark despite that being evidence against his existence (presumably because there's some other evidence that's overwhelming).
Here are my questions:
I am looking forward to your thoughts.