Rational vs. Scientific Ev-Psych
12Peter_de_Blanc
7Doug_S.
0HalMorris
1Alex_Kennedy
0HalMorris
9Nathan_Myers
8bigjeff5
4Eliezer Yudkowsky
1James_Bach
8JulianMorrison
3Dojan
15Virge2
3zslastman
1Peiter
5mnuez
5Nick_Tarleton
0martino
0outeast
4Carl_Shulman
0David_Pinto
11DSimon
1Caledonian2
0anonymous12
4Silas
0Dojan
1Caledonian2
1michael_vassar3
3Recovering_irrationalist
0Recovering_irrationalist
7Pseudonymous2
2poke
2Rolf_Nelson2
0poke
0Nathan_Myers
2Brian_Jaress2
0Nathan_Myers
3bee
0bee
7James_Barlow
3BlackHumor
3BlackHumor
0Kenny
2hylleddin
4Khaled
-8Sanguinaura
11TheOtherDave
1Tsujigiri
0John_Maxwell
4[anonymous]
1VeryPeeved
New Comment
Science is a special kind of strong evidence, a superset of rational evidence; if I say that my socks are currently white, you have a rational reason to believe, but it is not Science, because there is no experiment people can perform to independently verify the belief.
That would make it a subset, rather than a superset.
"The sexual division of labor in hunter-gatherer societies - "Men hunt, women gather" - is a human universal among hunter-gatherers, like it or not"
I think I've heard that the division isn't quite as sharp as all that (for example, that women would hunt targets of opportunity while gathering), but I can't remember exactly where...
There is quite a bit of movement in hunting, and frequently of flushing out the animal. When you see or hear the animal, then you might freeze. Did a twig snap in the refrigerator?
If you do not have sight of your quarry but you suspect it to be in the area then you'll probably freeze. Animals, humans included, are excellent at spotting motion. Of course, both of our ideas are essentially baseless, I think we'd have a clearer picture if someone performed Eliezer's study (or similar).
And if you startle a cat when he's licking his crotch he'll freeze in whatever awkward posture he's in (that's my overgeneralization from one male cat).
Certain observations about scientists' collective behavior long mystified me. Given an established theory that explains old evidence and but not new evidence, and a new hypothesis that also accounts for some new phenomena, scientists routinely demand (and defend demanding!) much more rigorous testing of the new hypothesis than the old theory ever withstood, before they will even accord it parity with the old. Also, when new data actually falsify an old theory, they will go to great lengths to try to rescue it, inventing no end of ad-hoc epicycles. Data that contradicts all known theories is carefully ignored.
My just-so story is that scientists are self-selected as people who Need to Know. To go from one established theory to two alternatives would be to step from knowing to uncertainty. To abandon a falsified theory would be to step from knowing to not knowing. Either of these is intolerable to one who Needs to Know. As a consequence, science is riddled with established, falsified old theories, and routinely ignores both viable new hypotheses and new data that doesn't fit any hypothesis.
If you need concrete examples, consider the quasar (a high-redshift light source) discovered in NGC7319 (an opaque low-redshift galaxy, one of Stephen's Quintet, in Pegasus) a few years back, and since very studiously ignored. Or, consider the standard theory of comet formation, flatly contradicted by every observation, but still trotted out in textbooks and NASA press releases. Each new press release expresses hope that its observations may someday help illuminate dark matter and dark energy, which thus far exist solely to rescue otherwise falsified models of galactic and supergalactic dynamics and cosmology.
Regarding the quasar "in" NGC7319 - read here for a discussion on the stellar object.
Needless to say, the reason it is ignored isn't because it contradicts mainstream cosmological theory, it's because it coincides with it pretty much perfectly given the state of our observational technology.
When judging articles like this, a simple, lazy estimate of probabilities is very helpful. For example, which do you think is more likely, 100 years of highly successful physics and cosmology is flat wrong (which is required for the quasar to be inside NGC7319, everything from what quasars are made of to the physics behind how they emit EM radiation must be wrong), or that a handful of researchers jumped to conclusions in their excitement? My vote goes to the latter until some pretty extraordinary evidence is found confirming their claims. Modern telescopes aren't precise enough to rule their claims out completely, but the quasar doesn't deviate from known cosmological effects which fit the current theory of the cosmos.
I'd wager a similar story for comets, I didn't look that up though as they don't interest me nearly as much as quasars do.
There is also a fairly significant amount of rather difficult to refute evidence for the existence of dark energy and dark matter. Matter that both has mass (and therefore gravity) and also does not interact with ordinary matter certainly exists - the best example is the neutrino. There just isn't a known mechanism to produce enough of this type of matter to account for the amount observed in the universe. Hence the "dark" in the name (which, incidentally, many physicists don't approve of precisely because it sounds too much like "magic").
The existence of dark energy is even more definitive. Simply put, the universe is expanding at an accelerated pace. This is undeniable. The only way for acceleration to occur is for energy to be expended (unless you are willing to throw away thermodynamics' otherwise perfect record). There are a couple of potential sources for dark energy (like vacuum energy) but nothing close to definitive.
Again, which is more likely? That hundreds of years of science is completely wrong (with nothing even sensible to take its place, I might add) or a hitherto unknown aspect of the universe has been discovered? Honestly, given the amount of evidence supporting the current models used in physics "something new" is by far the simplest and most likely explanation.
Dark matter and dark energy aren't stop signs to cosmologists and physicists, they are basically shorthand for "we don't know yet what this stuff is or how it got here, we are still trying to figure that out, but whatever it is should look something like this".
And if you think scientists have been ignoring these problems, especially dark matter, then you haven't been paying enough attention to theoretical and experimental physics over the past decade, particularly the past 5 years or so. One of the driving forces behind the LHC (certainly not the only, of course) was dark matter.
I think the advocates of Naturalistic Inquiry (see Lincoln and Guba) would say that you aren't talking about all of science, but of just the "positivistic paradigm" of science, whereas there is another paradigm called "naturalistic" or "constructivist" that does science differently.
I don't buy the whole Naturalistic program, but they raise some useful points. One of them is that the experiments you suggest require you to impose upon the object of your study an ontology along with the value system associated with it. When studying complex and ill-defined systems, such as psychological or social systems, this may suppress or disrupt the very phenomena that matter, and we never the wiser.
A naturalistic approach to science may tactically employ the kind of experiments you suggest, but proceeds with a great deal of caution about potential variables and hypotheses. "Hunter" and "gatherer" are socially overloaded terms with many implications and connections to other aspects of human life. It may be a big spaghetti mess to disentangle the issues. Inquiry proceeds in an exploratory fashion to tease out potential factors.
On other hand, it may not be a big mess! But the naturalistic bias is toward assuming complexity and subtlety and looking closely at the role of a priori assumptions in the choices of words, variables, and instrumentation that may lead to false results. It's sort of post-modernism applied to scientific method.
Again, I'm not a partisan of Naturalistic Inquiry. I just find it intriguing, and I have an allergic reaction to oversimplification (having been fooled so often by my own simplifications).
I think the heuristic people are using is not affect, but rather "blowing smoke" versus "a hypothesis specified in enough detail to test". They are drawing evidence from the implied detail, because it's harder to invent a detailed false theory without stumbling across an obvious counterfactual.
Most urban legends have lots and lots of detail, and many futurists add lots off specifics to their visions of the future, which make them seem more plausible. This has been elaborated on before.
[Edit: Spelling]
"This pattern of belief is very hard to justify from a Bayesian perspective. It is just the same hypothesis in both cases. Even if, in the second case, I announce an experimental method and my intent to actually test it, I have not yet experimented and I have not yet received any observational evidence in favor of the hypothesis."
Always consider the source.
It is the same hypothesis, but if the person telling the story is unknown to me at the outset, then the credibility of the source rises with test plans, as follows.
After the just-so story: This could be any crank making up stories to sell an idea.
After hearing of a possible test: If this person has taken the time to work out how they would test it, then my confidence is significantly increased. They've just set themselves apart from the pure story-tellers.
After looking at the thoroughness of their proposed tests: Confidence rises if they've shown a willingness to consider lots of ways that their story and testing could be wrong or misleading. Evidence of familiarity with and adherence to scientific methodology does increase credibility.
After hearing that they are committed to performing their tests: Confidence increases a little more. Not much, but a little.
Sure, these increases in confidence are small--tiny compared to the boost from actually performing even one of the tests--but they are not zero.
If I'm already familiar with the story-teller, and have some prior opinion of their experience and capability, then you're right: hearing about the tests shouldn't affect my confidence.
1I underwent the exact change in confidence described in the above article and for pretty much those reasons, on reflection. The situation is an extremely common on in science. The only place you're wrong is in saying that the confidence update is necessarily small. If somebody has worked out a series of experiments to test an idea, it means they've thought about the idea and taken it seriously, and are willing to advertise this at the risk of some of their credibility. You'd assign it a higher prior to such an idea than an off-hand remark Not even counting the demonstration of capability it provides.
I agree with Virge. Evolutionary "explanations" of human behaviour abound, especially in the mainstream media. More often that not, these make no particular sense and seem to have been pulled out of someone's backside. For example, many ignore that the traits in question have to increase viable offspring to have an evolutionary effect. Therefore, it is simply reasonable to consider the source.
No psychics have ever performed under scrutiny. Therefore, a priori, claims of psychic powers are met with skepticism. And reasonably so, I'd say.
Svara.
In dealing with the demons that haunt rationality among those who assume that they are in fact being rational I find no word as helpful as "svara".
I'm going to guess that you grew up pretty Modern Orthodox and therefore never swam and drowned in the sea of svara that chokes the souls of Chareidi youth. You see, in the word of "Iyun" ("In Depth") Talmudic study, the Talmud is NOT the text that occupies anyone's focus. The Talmud is just a small step ahead of the Torah in being a source text from which to jump into further discussions. An hour or two might be spent on studying a Talmudic passage after which a good month will be spent on the commentaries upon the commentaries upon the commentaries on the Talmud. Knowing the Talmud is nice, but no more impressive than knowing how to read - it's a simple means to an end, and one that's taken for granted.
The focus during the subsequent 14-hour-a-day "Talmud Study" is on explaining why a particular commentor holds a particular opinion regarding a particular law when - in a different volume - he holds an opinion that would seem to be at odds with the one that he expresses here. And after much shouting and pulling of hair the "Svaras" (loosely: "reasonings") are set on the table.
"The reason why he claims that a table that fell on a horse that fell in a pit full of imaginary gas is still a table that cannot contract impurity, while in the other case he says that a man with two penises is required, according to Rav, to have only the larger one circumcised (laws that - when analyzed - appear to be based in contradictory premises) is because he holds that a penis is not created until the fourth month which would imply that we can assume that a boy is a girl until that time which means..."
yah. That's the svara. It's the ABSOLUTELY BRILLIANT and intellectually beautiful way of explaining how Rabbinic opinions that appear to be inconsistent are actually FULLY consistent and priorly thought-out to a degree that borders on the incomprehensible.
And brilliant it often is, but bullshit it almost always is as well. There are NO TWO Rabbinic opinions (of the Rishonic Age - say 950 CE to 1430 CE) put forth by the same Medieval Rabbi that modern Yeshiva scholars would be hard-pressed to find a seemingly brilliant svara for. It's like Nostradamus (or Evolutionary) explanations (and pardon the sacrilege there, but ON THIS SUBJECT they're often, sadly, comparable) - except performed with mental gymnastics on a far higher sphere.
And it's almost always BS.*
mnuez
*Lest Iyun-Learning have any defenders here let me point out that its entire fundamental premise - it's foundation stone, if you will - is based on the premise of Rabbinic Infallibility. It's based on the belief that what Maimonides wrote when he was a young adult was letter-perfect and that his opinions/reasonings/traditions are expected to be identical to what he wrote in his final decade.
Which is naturally preposterous on account of the fact that Maimonides was human, he lacked modern archival resources, he STATES CLEARLY on a number of occasions that people who focus on Every Single Word in a text (even in the Bible!) - and expect brilliance therein - are fools, he writes about his own past errors and ask people to point out others that he might have missed and - did I mention? HE WAS VERY VERY HUMAN, and thus, likely NOT to be infallible. - Yah, and if HE wasn't a letter-perfect God, you can damn well be sure that his less methodical, brilliant and intellectually honest colleagues likely were not as well.
For example, many ignore that the traits in question have to increase viable offspring to have an evolutionary effect.
But they don't; they have to increase inclusive fitness, which could mean helping close relatives. (Your main point still looks right.)
I am not sure if your analogy works as you are making only one interpretation of the change in status from "just-so conjecture" to "plausible testable hypothesis".
This move could also provide explanatory power to the listener - not just that it is now plausible science (but not actually yet performed) - but that you have helped them see how the "hunter motionless for his prey" and "the frozen male fridge user" could be operating under similar psychological (or evolutionary or both) antecedent conditions. This is, of course, the whole point of your insight in the first place but because it now, ex post, appears obvious to you does not mean this is so in the listener. Your plausible hypothesis is an additional explanation to help them understand your argument better. In particular, you have shown with your move how your conjecture could be wrong and this is additional information and also contrary to "just-so" arguments. So the change in Bayesian weighting could come from better comprehension rather than because of the label "science" being attached.
You need a way of differentiating these two interpretations but, unscientifically :-) I agree with the your main point, making something appear scientific is a rhetorical move and should not alter one's expectations, just find a better example!
Funny that you assume that stating a testing methodology makes it sound more plausible. My own response was more like, 'well, OK: but my money would be on your disproving your hypothesis.'
The fact that someone is subjecting their view to a test and the possibility of being disproved does give us new information, like finding out that someone has bet on their view in a prediction market. The person has to have enough familiarity with scientific habits of thought (generally a positive sign for intelligence and rationality) to come up with the idea, and people are much more cautious about advancing theses where they fear being quickly proven wrong.
I believe can scientifically prove socks are white. You just need to measure the wavelengths of light reflected off of them.
I just tried it, but it turns out my socks are black. Therefore my replication failed. Though perhaps I just experienced a black swock effect.
In order for a hypothesis to NOT qualify as scientific, it cannot be testable. If we can figure out a way it can be tested, even if we can never accomplish the test ourselves, it's still testable.
If a hypothesis is not even potentially testable, there cannot be a difference in the implications of the affirmation of the hypothesis and the negation of the hypothesis.
If there is no difference in the affirmation and negation of the hypothesis, the hypothesis conveys no information. It doesn't mean anything at all.
The relationship between "expected evidence" and hindsight bias ought to be carefully considered. Since the set of plausible "just-so" explanations is effectively infinite, it's quite difficult to anticipate them in a formal decision model and assess their probability. You are focusing on a tiny fraction of explanations for which we can come up with testable evidence: this is not a convincing argument.
But if you can explain inconsistent events with the same theory, that is not scientific. For example, saying that evolutionary psychology explains why women do and do not act sexually provocatively when fertile, because that helps them to produce offspring, and exposes them to the risk of rape at a bad time.
Good point!
One have to be open for the possibility of different tactics in different situations though.
If you include the perceived quality of potential sexual partners in the calculation, that could turn Silas' example into a scientific one.
But he's absolutely right with the data as given.
Nathan: A new theory that matches old data is likely to be more complex than an old theory that matches old data because the old theory is probably the first that someone thought of and bothered publishing. This justifies some conservatism, but not so much. The effort required to revise beliefs or to hold them as a Bayes net is a further justification, but still justifies much less conservatism than is seen.
OTOH, I disagree with your dark matter example. The case for dark matter over weird physics actually seems fairly compelling to me, though this probably isn't the place to discuss it.
Even if, in the second case, I announce an experimental method and my intent to actually test it, I have not yet experimented and I have not yet received any observational evidence in favor of the hypothesis.
But at least you've thought it through from an experimental method point of view, and not noticed a serious defect in your theory in the process. As long as a false theory is less likely to survive that "test", it counts as some bayesian evidence in favor.
Well, fine, but I bet it's still true.
I bet your mother was older than you and had therefore had more time to learn not to do daft things such as keep re-searching the refrigerator from exactly the same angle.
But there is no such thing as a theory of age related psychology* to bring that explanation to mind, and would not have been a very flattering theory to come up with.
*Perhaps there should be.
There's two interesting things going on here:
(1) You rescued your just-so story about the evolution of your behavior with a just-so story about how it could be scientifically established. But the same argument that knocks down your evolutionary just-so story knocks down your sociology of science just-so story; neither contains any established truth, both are products of your imagination.
(2) Science is exemplary human knowledge. If our account of how science proceeds tells us science should have proceeded differently, we cannot blame the scientists, we must blame our account of science. If Empiricism states that scientists are doing something wrong; so much the worse for Empiricism. If falsificationism states that scientists are doing something wrong; so much the worse for falsificationism. If Bayesian theory states that scientists are doing something wrong; so much the worse for Bayesian theory. Since no example of successful scientific practice in the history of science has ever relied on anything with even the slightest bit of resemblance to Empiricism, falsificationism or Bayesianism; so much the worse for them all.
Is there a word for the similar case to the "just-so" story, but that has a spurious environmental explanation rather than a spurious genetic explanation? (For example, "boys are more aggressive than girls because parents give their boys more violent toys.") I see many more of the former than the latter in the media.
Rolf Nelson,
I would just call that an example of folk psychology. Note that I can take your example, think up a bunch of experiments like Eliezer did, and argue that it's now science. (This is done; people look for statistical correlations between violent videogames and aggressive behavior. But you can come up with the equivalent of Eliezer's eye tracking experiments; just measure physiological correlates of heightened aggression while you have boys play with toy guns.) I think that would be crappy science.
Michael V: Dark matter exposes another sort of bias common among scientists. A wide variety of anomalous (or once-anomalous) astronomical phenomena are consistent with plasma fluid-dynamic phenomena at various scales ("geysers" on Enceladus, the Aurora Borealis, solar and galactic jets, too-fast galactic rotation). However, the mathematics of plasma fluid dynamics is fiendishly difficult, and effectively intractable. Progress is possible by performing tricky vacuum chamber experiments, and by simulations on very large supercomputers. Astrophysicists, though, are self-selected from among physicists who don't care for laboratory work, and who enjoy clean mathematical derivations.
(N.B.: Plasma fluid dynamics involves no exotic science at all; it's all just time- and space-varying electric and magnetic fields and (sometimes relativistic) particle flows.)
Parsimony demands that electromagnetic effects be shown to be insufficient before trotting out completely new and property-less "dark" particles and forces. Furthermore, anybody invoking exotic forces should still be obliged to account for the asserted lack of effect from the millions or quadrillions of tons of plasma acknowledged to be in motion in the systems observed.
A proper accounting would need astrophysicists to learn a new field, so of course it won't happen. (One who did would never get papers using it accepted.) Instead, astrophysicists give one another a pass. The rest of us, knowing, may chuckle at their press releases. In the meantime, astrophysicists collect huge masses of detailed data from wonderful new instruments, but remain unable to synthesize it into anything even remotely plausible.
Has this bias exhibited by astrophysicists been named yet? Of course its analog is practically universal outside of science: "to do that I would need to learn, um, math".
"There are some people who will, if you just tell them the Refrigerator Hypothesis, snort and say 'That's an untestable just-so story' and dismiss it out of hand; but if you start by telling them about the gaze-tracking experiment and then explain the evolutionary motivation, they will say, 'Huh, that might be right.'"
But do they actually think it's more likely to be true?
They didn't say it was impossible, they said it wasn't testable. Explain how to test it, and they don't say that. What's the problem?
Michael V: I hope you can offer a brief hint by what criterion "dark matter" might be distinguished from "weird physics". (I suspect it will turn out to be very much on-topic, for the site if not the thread.)
Two points ...
1) I had the same experience as a teen ... the only difference being that my dad said and did what your mother did ... you are seeing patterns where none exist
2) From an evolutionary perspective your inference is not correct or valuable ... note first point and read Cosmides
Two points ...
1) I had the same experience as a teen ... the only difference being that my dad said and did what your mother did ... you are seeing patterns where none exist
2) From an evolutionary perspective your inference is not correct or valuable ... note first point and read Cosmides
This is testable isn't it?
You'd need a fridge, some ketchup plus some miscellaneous condiments, and a clip board and white coat of course (since otherwise people would think you were a nutter). Set up in a public space and ask men and women to "find the ketchup". Observe their movements and note the point at which they start "hunting" behaviour. Bit of analysis for men versus women, maybe add in control trials where there is no sauce in the fridge, and hey presto - you PhD is assured.
If you want to get really posh, link a sensor to the fridge door to start the time trial and get the subject to stand on a Dance! Dance! revolution mat so you can see how their weight shifts.
Same general counterargument as the other people who've posted:
1) If this anecdote is all you have to base your theory on, you have essentially no more chance of being right than I would be making up random theories in quantum mechanics.
2) If you say "I think men can find jars easier because male hunter-gatherers hunted", you are likely some random crank who has just enough experience in the field to think of the idea. Once you suggest a method to test it, you prove that you are familiar enough with the idea and with the rest of the field to know what would prove it which elevates you from "some random crank" to "guy with a strange idea".
Also, though Wikipedia is not an entirely reliable source, it contradicts your claim that "men hunt women gather" is a human universal. Though it's more common than not, there are a few hunter-gatherer tribes where women help men track animals, or where men also gather sometimes, and at least one tribe where women also kill the animals.
Not that you're likely to read this, of course, since you posted the OP years ago, but I just thought I should further point out that your theory is very improbable.
When you described your possible experiment, you raised the probability that your theory is provable/disprovable, hence you raised the probability of it being rational.
Your desciption didn't raise the probability of your theory being correct, it raised the probability of it being a theory!
It does seem possible that hypotheses which are relatively easy to test tend to be more likely to be true, either for theoretical reasons or just because of human nature. For example, let's say my friend believes in God, but their hypothesis is sufficiently complex to defy easy testing--praying for miracles doesn't work, evil exists in the world, etc. all for various complicated reasons my friend explains to me.
The real moral is: just remember where you put the damn ketchup.
Also, this idea ties neatly into the idea of hypothesis space. If you're going to put forward a hypothesis at all, you'd better have good reason to pick THAT hypothesis over the approaching-infinity other's. The effort to draw a single, specific hypothesis out of the plethora of potential hypotheses is going to be greater than the effort to show that specific hypothesis to be probable.
"He is frozen, waiting for the ketchup to come into sight." might just be the funniest sentence I've seen on this website so far.
Curated and popular this week