= 27d567bc2d48d9f40e9ccc20ea370b15)
Tsujigiri comments on A Parable On Obsolete Ideologies - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (272)
As the word evidence is commonly used, there is evidence for Russell's teapot -- just not evidence that you or me believe in. If someone says "Russell's teapot exists! I've seen it!", that is anecdotal evidence for its existence. Anything that suggests something is true or false is evidence, no matter how flawed that evidence may be.
It is by considering all the evidence, for and against our beliefs, that we progress towards truth.
I think I might have to write something specifically addressing this misconception because a few people seem to have picked it up.
It's only evidence for the existence of Russell's teapot if more people say they have seen it than you would expect in a universe where Russell's teapot does not exist.
(That's ignoring the fact that Russell's teapot is by stipulation non-observable and hence in that artificial situation we can skip Bayesian updating and just go straight to p=1 that anyone claiming to have observed it is lying or deluded).
I think our disagreement is to do with our differing usage of "evidence", not a misconception. I'd say that a sole anecdote of someone seeing Russell's teapot can be considered evidence for its existence, even though it's not credible evidence.
I would add that different situations require different standards of evidence, depending on how willing we are to accept false positives. The fire service only requires one phone call before they respond.
Perhaps a less abstract example would help. Uri Geller used to claim he was projecting his psychic powers over the TV or the radio and invite people to phone in if something "spooky" happened in their home. Inevitably people phoned in to report clocks stopping or starting, things falling off shelves and so on. It was pretty convincing stuff for people who believed in that sort of thing, but it turned out that when skeptics with no psychic powers whatsoever pretended to be psychic on the radio and invited people to phone in they got exactly the same flood of calls. Odd things happen all the time and if you get a large enough sample of people looking about the house for odd things to report you get a fair number of calls.
What would have been evidence for Uri Geller having psychic powers is if he got more calls than normal people when he did that stunt. Just getting the base rate of calls anyone else would proves nothing. As you said yourself, you have to look at all the evidence.
You might be thinking "I read this Bayes theorem essay, and it said evidence for X was whatever was more likely to be true in a universe where X was true. In a universe where Russell's teapot existed I'd be more likely to hear someone say they saw Russel's teapot, right? So it's evidence! Bayes says so". That line of reasoning only works if you don't have all the evidence to look at so you can't determine the base rate. If you can determine the base rate then it's probably going to turn out that the number of claimed teapot-sightings is consistent with the base rate of stupid noises humans make.
If all you have is the one anecdote then it does count as evidence, but only in a strictly philosophical or mathematical sense. Not in any practical sense though since the shift in the relevant p value isn't going to be visible in the first twenty or thirty decimal places and I doubt anyone alive has that level of precision in their decision-making. (The odds of someone having seen Russell's teapot could be said, very conservatively, to be lower than the odds of someone winning the lotto twice in a row).
I don't think a less abstract example will solve a dispute over word usage.
And even if they did receive a statistically significant number of calls, perhaps people lied, grouped together and phoned in supernatural events that hadn't actually occurred.
I don't have a problem with dismissing evidence if it truly is highly irrelevant. I think the problem is that in some cases evidence that at first seems irrelevant turns out to be important.
How relevant a piece of evidence truly is might not become apparent even after significant consideration. This is why, for important matters, I explore as much evidence as possible, even the seemingly irrelevant evidence.
This is really a dispute over maths. The laws of probability are the law, they don't depend on word usage.
Of course people can mock up evidence of things that are not in fact true. It's still evidence from the perspective of people who are not in on the plot. All of our evidence for everything could in theory have been mocked up by the architects of the Matrix and it would still all be evidence, from our perspective.
I'm not clear how this is relevant to the base rate fallacy though.
Strictly speaking what's going on there is that you are collecting facts which might later turn out to be evidence for a hypothesis you have not articulated yet. This is an entirely rational approach to solving certain classes of problems. It doesn't change the definition of evidence though,
Explaining the way one uses a word isn't a statement about maths or the laws of probability either.
It's not. I was riffing on what you said.
I was discussing ascertaining the trustworthiness of evidence concerning a hypothesis you are currently considering. Like an investigation into whether Uri Gellers phone calls were genuine reports of supernatural events, for example.
Of course not. I'm not trying to suggest that my usage of "evidence" is somehow better or superior than yours. I do think mine is more common, but that's a matter of opinion.
It is, actually. It's the Bayesian definition that evidence for X is something more likely to be true in a universe where X than in a universe where -X.
Part of the point of Bayes' Theorem is that correction for the base rate fallacy is baked in. That's a large part of what separates Bayesian rationality from irrationality,
What you're saying here is that you use Bayes' theorem to inform your definition of "evidence".
If I used a different definition of evidence, that doesn't mean I'm saying something about Bayes' theorem. That simply means I use the word differently.
When it comes to evidence, I don't believe Bayes' theorem deals with the real-world problems that arise when considering a hypothesis. For example, it doesn't deal with the "garbage in, garbage out" problem.
As I said, we might plug the number of calls Uri Geller got into Bayes' theorem and because of the answer believe that supernatural events did actually occur. But that would be an incorrect conclusion because we have based our conclusion on the faulty premise that more calls means supernatural events are more likely to have occurred.
If you mean things like the base rate fallacy, then yes it does. If you mean that putting in random numbers for your priors doesn't solve your problems, then there isn't any method of considering evidence that fixes that in principle.
Absolutely. Otherwise I can't exclude from the domain of "evidence of X" all things which should not incline a rational person to amend their views about X, and I very much want to do that.
Note that something can be "evidence" without being "evidence of X" where X is one specific something. Someone calling in to say their watch stopped is evidence that someone called in and evidence that a watch stopped and so on, just not evidence that Uri Geller has radio-propagated psychic powers.
I agree with you to the extent that Bayes' theorem is not a magic wand that cures all epistemological ills. You don't have to browse this site for too long to come across people advancing silly ideas under the banner of Bayesian inference because they are using it incorrectly. However while it's not a magic wand it's a mathematical truth about how the universe works, and any time you deliberately deviate from Bayes' theorem you are deliberately going wrong as a matter of mathematical fact.
What is actually going on here, assuming you are applying Bayes correctly, is that the prior probability of Uri Geller having radio-propagated psychic powers should be seen as astoundingly low. Far lower than the odds of me winning the lotto twice in a row, for example - let's say one on ten to the fourteenth as a very generous prior probability. If Uri Geller got a lot more phone calls than a skeptic pretending to be a psychic that should tip the scales a little in his favour but nowhere near enough to get P(Uri Geller has psychic powers) up to a level we should take seriously.
The premise "more calls makes psychic powers more likely" is not flawed at all. If we lived in a bizarre universe where Uri Geller really could reach out through your radio and stop your watch then we would indeed see more calls coming in, and if we lived in that bizarre world we should want to believe we lived in that bizarre world. I'm very sure we do not live in that world but that's because the evidence is against it, not because I would dismiss that evidence if it supported it.