Bayesians must condition their beliefs on all available evidence; it is not cheating to use less than ideal sources of information. However, this process also requires conditioning on the evidence for your evidence. Outside of academic journals, evidence is often difficult to trace back to the source and is dependent on our notoriously faulty memory. Given the consequences of low-fidelity copying, should rationalists trust evidence they can't remember the source of, even if they remember reading the primary source themselves? Should community members be expected to produce citations on demand?
This issue came to mind while trying to find a study I vaguely remembered about how the increased happiness of the religious could be explained by increased community involvement and while trying to factcheck PhilGoetz's now infamous anecdote about Steve Jobs. I started contemplating the standards for relaying highly relevant, but potentially wrong or distorted information.
Luckily factchecking is much easier in the age of the internet. Wikipedia serves as a universally accessible standard reference, and Google serves well for everything else. But sometimes my google-fu is not strong enough. So, I'll put this to the community: how should rationalists balance the tradeoff between neglecting evidence and propogating bad information?
Hygienic practices have been touched on before, but I haven't seen any consensus on this issue. Are the standards for what you personally condition on and what you share in discussion different? What needs a citation and what doesn't? Does anyone have recommendations for ways to better track the sources of evidence, i.e. reference management software?
I assume you mean what to do in conversation and in comments on Less Wrong. The standards for things you put in print are that everything be supportable (this is never achieved throughout an entire book, but it is the goal).
This is difficult for me, because my memory is so poor that I can rarely remember the reasons for my conclusions, let alone the data behind those reasons, let alone the sources of that data. If I can remember my conclusions, I think I'm doing pretty well.
The best that I can do is be upfront about it when I can't remember the data. This is harder to do than it sounds. It's easy to blurt out a statement without checking whether you can support it. I did this just 2 days ago, when someone asked me about the precision and recall figures being claimed for a piece of software I had worked on. They sounded too high to me, and I said I didn't believe them. Hours later, I realized that although I had a feeling those figures were too high, I had no memory for any specific precision and recall figures. So tomorrow I need to talk with that person again and give them a disclaimer.
My approach is to say what I think, but give disclaimers when I can't remember supporting information. The amount of disclaiming depends on the audience. If I'm telling a child to eat his vegetables, I'm not going to give him a disclaimer saying that I can't actually remember the nutritional content of broccoli.
I do a lot of inferential reconstruction. For instance, the Steve Jobs anecdote is from Steve Jobs: The Journey is the Reward. I say this not because I specifically remember it being in that book, but because that is the only biography of Steve Jobs that I remember reading.