= 27d567bc2d48d9f40e9ccc20ea370b15)
byrnema comments on Guilt by Association - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (38)
"If P, then Q. Q. Therefore, P."
I have taught this as a fallacy for the last two semesters in a propositional logic section of my course. However, given the prevalence of this fallacy, as well as the observed resistance of students to assimilating it, I would like to propose that this really isn't a fallacy but a disconnect between what we often naturally mean by "p implies q" and what it technically means in propositional logic.
I argue that what we tend to understand by the statement "p implies q" is that p is the set of things that result in q. This can be tested by the fact that if Q is known to have two causes P1 and P2, and someone says that P1 implies Q, it would be rather natural and logical for another person to "correct" them by adding that P2 also causes Q. But the word correct is too strong ... it is rather an augmentation.
I think that, like many statements in ordinary usage, "P implies Q" is a fundamentally Bayesian statement; a person means: "in my experience, P is the set of things that cause Q". The 'in my experience' means that it is possible (but not observed) that Q might not happen after P, and also that other things besides P might cause Q, but again this hasn't been observed or isn't recalled at the moment. A person is normally willing to be flexible on both points in the face of new evidence, but is taught in logic class to be absolute for a moment while considering the truth or falsity of the statement.
So finally, when an untrained person is told that "p implies q" is a true logical statement, in their efforts to be "absolute" they may think that both implied meanings are true: p always causes q AND only p causes q. It takes some training (deprogramming of what it has seemed to mean their whole natural lives) to understand that only the first is intended (p always causes q) when the statement is "True" in formal logic.
Calling the logical arrow that means "not P or Q" by the name "implication", even if you say "material implication", might've been a bad idea to begin with.
Perhaps a better example of a "fallacy" that is really just a mismatch between expected verses actual meanings is the example of how many doctors fail to accurately estimate the probability of false positives given a positive. It's called the base rate fallacy and works like this (from here):
Given that a women has a positive result, what is the probability that she actually has breast cancer?
It turns out that the probability she has cancer is only 7.76%, but many doctors would over-estimate this.
Well, I argue that this is a mismatch between the intellectual way a mathematician would present the problem and the way a doctor experiences medicine. A doctor who frequently gives mammograms would certainly learn over time that most women who have a positive result don't have breast cancer. From their point of view, the occurrence of 'false positives" -- i.e., results that were positive but false -- is 92%. Yet on a pen and paper test they are told that they rate of false positives is 9.6% and they misinterpret this.
On the one hand, you could just explain very clearly what is meant by this other rate of false positives. Doctors are generally intelligent and can understand this little circumlocution. On the other hand, you could instead give them the more natural figure -- the one that jives with experience; that 92% of positives are false, and remove the fallacy altogether.
I think that mathematicians have an obligation to use definitions that jive with experience (good mathematics always does?), especially instead of calling common sense "fallacious" when, actually, it is just being more Bayesian than frequentist.
The page you link includes the "9.6% false positive" usage, but that terminology is preceded by,
making the interpretation of the phrase clear.
The mismatch isn't intellectual versus experiential in the way you claim. Most people get the problem right when the numbers are stated as frequencies relative to some large number instead of probabilities or percentages, i.e., when the wording primes people to think about counting members in a class.
It's still pretty scary that doctors would have to be primed to get basic statistical inference right (a skill that's pretty essential to what they claim to do). The real world doesn't hand you problems in neat, well-defined packages. You can guess the teacher's password, but not Nature's.
After I got into a warm discussion with some other members of the speech and debate club in high school, I started doing a little research into the field of medicine and its errors.
Long story short: doctors are not the experts most people (including many of them) believe them to be, our system of medicine is really screwed up, and it's not even obvious that we derive a net benefit from medical intervention considered overall.
(It's pretty obvious that some specific interventions are extremely important, but they're quite basic and do not make up the majority of all interventions.)
I was about to lecture you on how wrong you are, until I realized I've never encountered a counterexample.
Please note that I do not rule out the possibility that we derive a net benefit. It's just that it isn't obvious that we do.
A counterexample of my being right? Or a counterexample relating to medicine?
As in, "I have never encountered a doctor that actually understood the limits of his knowledge and how to appropriately use it, nor a clinical practice that wasn't basically the blind leading the blind."
Okay. I was unsure if your statement was meant to be a personal insult or a comment about medicine - your comments have cleared that up for me.
If I may offer a suggestion:
Access NewsBank from your local library, go to the "search America's newspapers" option, and do some searching for the phrase "nasal radium". There will be lots of duplication. You may find it useful to only search for articles written between 1990 and 1995, just to get a basic understanding of what it was.
Then realize that the vast majority of surgical treatments where introduced in pretty much the same way, and had the same amount of pre-testing, as nasal radium.
I don't infer doctors' actual performances from their responses to a word problem, so I'm not that scared. I don't think byrnema was wrong to claim that
Er, the whole point of statistical inference (and intelligence more generally) is that you can get the most knowledge from the least data. In other words, so you can figure stuff out before learning it "the hard way". If doctors "eventually figure out" that most positives don't actually mean cancer, that means poor performance (judged against professionals in general), not good performance!
"Eventually" was byrnema's usage -- I'd bet doctors are probably told outright the positive and negative predictive values of the tests by the test designers.
I see no disagreement. You are describing another way that the numbers could be presented so that it would be understood. I am not suggesting that doctors literally confuse the two ways of defining "false positive" but that the definition of false positive given is so far outside of experience, apparently, they are confused/mistaken about how to apply it correctly. My point is that if they actually needed it outside the exam once or twice (i.e., if the result was connected enough with experience to identify the correct or incorrect answer) they would readily learn how to do it.
You have asserted that the reason doctors can accurately tell patients their chances after a diagnostic test even if they perform poorly on the word problem is because they are confused about the term "false positive". But the problem can be phrased without using the word "positive" at all, and people will still get it wrong if it's phrased in terms of probabilities and get it right if it's phrased in terms of relative frequencies. So the fact that doctors can tell patients their chances after a diagnostic test even if they perform poorly on the word problem has nothing to do with them being confused about false positives.
But since that's not what the words mean even in standard English, it's clearly a misunderstanding on the part of the students.
Doesn't it depend upon the context?
Suppose the context is some event P. Then we can talk about what things are implied by P and P implies Q has the standard/technical logical meaning. (If P implies Q1, Q2 and Q3 we naturally but not logically expect all 3 in a "true" answer: P -> Q1 ^ Q2 ^ Q3)
On the other hand, if the context is Q, and we ask "what implies Q?" then we expect a fuller answer for P; P is the set if all things that imply Q: P1 v P2 v P3 -> Q.
Perhaps, generally writing P-> Q as (P v S) <=> (Q ^ T) would more accurately capture all intended meanings (technical and natural). It would be understood that S and T are sets that complete the intended sets on each side necessary for the "if and only if" and that they could possibly be empty.
(Alas, this would make it no easier to teach. I just stress in class that P implies Q means P is one example of things that imply Q, and Q is, likewise, need only be one example of things implied by P.)
No. "P implies Q", even in regular, everyday English, does not suggest that P is the set of all possible causes for Q. Context doesn't matter.
So I would guess you don't understand why people make the mistake that "if not Q, then also not P". Do you have another hypothesis for the origin of this mistake? (Perhaps there is more than one cause, ha ha.)
Later edit: The first sentence had an obvious error. In the quotes, I meant to write, "if Q, then P" -- or, more symmetrically, "if not P, then also not Q" as the mistake that is often made from "if p then q".
I'm actually in large agreement with you about what "p implies q" means in ordinary English, but can wobble back and forth with some effort. Let me try a little harder to convince you of the interpretation I've been arguing.
Let's suppose you are told, "if P then Q". In everyday life, you can usually take this to mean that if Q then P because P would have caused Q. If Q could instead have been caused by R and R was likely, then why didn't the person say so? Why didn't the person say "if R or P then Q"?
Um... I don't think that's a mistake. Given "If P, then Q", the non-existence or falsehood of Q requires that P also not exist / be false. It leads to contradiction, otherwise.
Seriously? P→Q ⊢ ¬Q→¬P.
Perhaps people are just not good at processing asymmetrical relations. They may naturally assume, for any relation R, that aRb has the same meaning as bRa. They may not notice that conclusions they make from the mistake at this level of abstraction contradicts their correct understanding at a lower level of abstraction that includes the actual definition of implication.
Interesting, but this doesn't seem true true in general. People are pretty good at not confusing aRb and bRa when R is something like "has more status than", for example.
Good point. When the relation is obviously antisymmetric, where aRb implies not bRa, this is enough to make people realize it is not symmetric.
I wouldn't be surprised if the easiest relations for us to imagine between two variables were simply degrees of "bidirectional implication" or "mutual exclusivity".
Bing bing bing!
The real issue, of course, is why they're the easiest for us to represent.
That's coming up next.