Prerequisite:  Evolutionary Psychology

Years ago, before I left my parents' nest, I was standing in front of a refrigerator, looking inside.  My mother approached and said, "What are you doing?"  I said, "Looking for the ketchup.  I don't see it."

My mother reached behind a couple of bottles and took out the ketchup.

She said, "If you don't see the ketchup, why don't you move things around and look behind them, instead of just standing and staring into the refrigerator?  Do you think the ketchup is magically going to appear if you stare into the refrigerator long enough?"

And lo, the light went on over my head, and I said:  "Men are hunters, so if we can't find our prey, we instinctively freeze motionless and wait for it to wander into our field of vision.  Women are gatherers, so they move things around and look behind them."

Now this sort of thing is not scientifically respectable; it is called a "just-so story", after Kipling's "Just-So Stories" like "How the Camel Got His Hump".  The implication being that you can make up anything you like for an evolutionary story, but the difficult thing is finding a way to prove it.

Well, fine, but I bet it's still true.

The sexual division of labor in hunter-gatherer societies - "Men hunt, women gather" - is a human universal among hunter-gatherers, like it or not; and it has experimentally tested cognitive consequences:  Men are better at throwing spears; women are better at remembering where things are.  So that part is pretty much nailed down - the controversial thing is saying, "This is why men stand motionless and stare into the refrigerator."

But even the Refrigerator Hypothesis is not as untestable as one might think.  For a start, you have to verify that this is a cross-cultural universal - you have to check to see if men and women in China and South Africa exhibit the same stereotypical behavior as in the US.  And you also have to verify that hunters who don't see prey where they expect it, do indeed freeze motionless and wait for the prey to wander across their vision.

But that doesn't prove that the same psychology is at work in a man staring into a refrigerator, does it?

Well, you could install an eye-tracker on a hunter; look for a characteristic pattern of eye movements when they're frozen waiting for prey; and use simulations or game theory to show that the gaze pattern is efficient.  Then you could put an eye-tracker on a man looking into a refrigerator, and see if they show the same gaze pattern.  Then you would have demonstrated a detailed correspondence which enables you to say, "He is frozen, waiting for the ketchup to come into sight."

Here is an odd thing:  There are some people who will, if you just tell them the Refrigerator Hypothesis, snort and say "That's an untestable just-so story" and dismiss it out of hand; but if you start by telling them about the idea of the gaze-tracking experiment and then explain the evolutionary motivation, they will say, "Huh, that might be right."  Because then, you see, you are proposing a Scientific hypothesis; but in the earlier case, you are just making up a story without testing it, which is very Unscientific.  We all know that Scientific hypotheses are more likely to be true than Unscientific ones.

This pattern of belief is very hard to justify from a Bayesian perspective.  It is just the same hypothesis in both cases.  Even if, in the second case, I announce an experimental method and my intent to actually test it, I have not yet experimented and I have not yet received any observational evidence in favor of the hypothesis.  So in either case, my current estimate should equal my prior probability, estimated the degree to which the "just-so story" seems "just" versus "so".  You can't revise your beliefs based on an expected experimental success, by Conservation of Expected Evidence; if you expect the experiment to succeed, then it was a rationally convincing just-so story.

People confuse the distinction between rationality and science.  Science is a special kind of strong evidence, a subset of rational evidence; if I say that my socks are currently white, you have a rational reason to believe, but it is not Science, because there is no experiment people can perform to independently verify the belief.

If you have a hypothesis you have not figured out how to test with an organized, rigorous experiment, then your hypothesis is not very scientific.  When you figure out how to do an experiment, and more importantly, set out to do the experiment, then your hypothesis becomes very scientific indeed.  If you are judging probabilities using the affect heuristic, and you know that science is a Good Thing, then making the jump from "merely rational" to "scientific" might seem to raise the probability.

But this itself is not normative.  Figuring out a way to test a belief with an organized, rigorous, repeatable experiment is certainly a Good Thing - but it should not raise the belief's rational probability in advance of the experiment succeeding!  A hypothesis may become "more scientific" because you are going to test it, but it doesn't get any of the power of scientific confirmation until the experiment succeeds.  To whatever degree you guess that the experiment might work - that it's likely enough to be worth performing - you must have arrived at a probabilistic belief in the hypothesis being true, in advance of any scientific confirmation of it.

I conclude that an evolutionary just-so story, whose predictions you cannot figure out how to test in any organized, rigorous, repeatable way, may nonetheless have a substantial rational credibility - equalling the degree to which you would expect a rigorous experiment to succeed, if you could only figure one out.