Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

An Especially Elegant Evpsych Experiment

23 Post author: Eliezer_Yudkowsky 13 February 2009 02:58PM

Followup toAdaptation-Executers not Fitness-Maximizers, The Evolutionary-Cognitive Boundary

"In a 1989 Canadian study, adults were asked to imagine the death of children of various ages and estimate which deaths would create the greatest sense of loss in a parent. The results, plotted on a graph, show grief growing until just before adolescence and then beginning to drop. When this curve was compared with a curve showing changes in reproductive potential over the life cycle (a pattern calculated from Canadian demographic data), the correlation was fairly strong. But much stronger - nearly perfect, in fact - was the correlation between the grief curves of these modern Canadians and the reproductive-potential curve of a hunter-gatherer people, the !Kung of Africa. In other words, the pattern of changing grief was almost exactly what a Darwinian would predict, given demographic realities in the ancestral environment...  The first correlation was .64, the second an extremely high .92."

(Robert Wright, summarizing:  "Human Grief:  Is Its Intensity Related to the Reproductive Value of the Deceased?"  Crawford, C. B., Salter, B. E., and Lang, K.L.  Ethology and Sociobiology 10:297-307.)

Disclaimer:  I haven't read this paper because it (a) isn't online and (b) is not specifically relevant to my actual real job.  But going on the given description, it seems like a reasonably awesome experiment.  [Gated version here, thanks Benja Fallenstein.  Odd, I thought I searched for that.  Reading now... seems to check out on the basics.  Correlations are as described, N=221.]

The most obvious inelegance of this study, as described, is that it was conducted by asking human adults to imagine parental grief, rather than asking real parents with children of particular ages.  (Presumably that would have cost more / allowed fewer subjects.)  However, my understanding is that the results here squared well with the data from closer studies of parental grief that were looking for other correlations (i.e., a raw correlation between parental grief and child age).

That said, consider some of this experiment's elegant aspects:

  • A correlation of .92(!)  This may sound suspiciously high - could evolution really do such exact fine-tuning? - until you realize that this selection pressure was not only great enough to fine-tune parental grief, but, in fact, carve it out of existence from scratch in the first place.
  • People who say that evolutionary psychology hasn't made any advance predictions are (ironically) mere victims of "no one knows what science doesn't know" syndrome.  You wouldn't even think of this as an experiment to be performed if not for evolutionary psychology.
  • The experiment illustrates as beautifully and as cleanly as any I have ever seen, the distinction between a conscious or subconscious ulterior motive and an executing adaptation with no realtime sensitivity to the original selection pressure that created it.

The parental grief is not even subconsciously about reproductive value - otherwise it would update for Canadian reproductive value instead of !Kung reproductive value.  Grief is an adaptation that now simply exists, real in the mind and continuing under its own inertia.

Parents do not care about children for the sake of their reproductive contribution.  Parents care about children for their own sake; and the non-cognitive, evolutionary-historical reason why such minds exist in the universe in the first place, is that children carry their parents' genes.

Indeed, evolution is the reason why there are any minds in the universe at all.  So you can see why I'd want to draw a sharp line through my cynicism about ulterior motives at the evolutionary-cognitive boundary; otherwise, I might as well stand up in a supermarket checkout line and say, "Hey!  You're only correctly processing visual information while bagging my groceries in order to maximize your inclusive genetic fitness!"

• 0.92 is, I think, the highest correlation I've ever seen in any ev-psych experiment, and indeed, one of the highest correlations I've seen in any psychology experiment.  (Albeit I've seen e.g. a correlation of 0.98 reported for asking one group of subjects "How similar is A to B?" and another group "What is the probability of A given B?" on questions like "How likely are you to draw 60 red balls and 40 white balls from this barrel of 800 red balls and 200 white balls?" - in other words, these are simply processed as the same question.)

Since we are all Bayesians here, we may take our priors into account and ask if at least some of this unexpectedly high correlation is due to luck.  The evolutionary fine-tuning we can probably take for granted; this is a huge selection pressure we're talking about.  The remaining sources of suspiciously low variance are, (a) whether a large group of adults could correctly envision, on average, relative degrees of parental grief (apparently they can).  And, (b), whether the surviving !Kung are typical ancestral hunter-gatherers in this dimension, or whether variance between hunter-gatherer tribal types should have been too high to allow a correlation of .92.

But even after taking into account any skeptical priors, correlation .92 and N=221 is pretty strong evidence, and our posteriors should be less skeptical on all these counts.

• You might think it an inelegance of the experiment that it was performed prospectively on imagined grief, rather than retrospectively on real grief.  But it is prospectively imagined grief that will actually operate to steer parental behavior away from losing the child!  From an evolutionary standpoint, an actual dead child is a sunk cost; evolution "wants" the parent to learn from the pain, not do it again, adjust back to their hedonic set point, and go on raising other children.

• Similarly, the graph that correlates to parental grief is for the future reproductive potential of a child that has survived to a given age, and not the sunk cost of raising the child which has survived to that age.  (Might we get an even higher correlation if we tried to take into account the reproductive opportunity cost of raising a child of age X to independent maturity, while discarding all sunk costs to raise a child to age X?)

Humans usually do notice sunk costs - this is presumably either an adaptation to prevent us from switching strategies too often (compensating for an overeager opportunity-noticer?) or an unfortunate spandrel of pain felt on wasting resources.

Evolution, on the other hand - it's not that evolution "doesn't care about sunk costs", but that evolution doesn't even remotely "think" that way; "evolution" is just a macrofact about the real historical reproductive consequences.

So - of course - the parental grief adaptation is fine-tuned in a way that has nothing to do with past investment in a child, and everything to do with the future reproductive consequences of losing that child.  Natural selection isn't crazy about sunk costs the way we are.

But - of course - the parental grief adaptation goes on functioning as if the parent were living in a !Kung tribe rather than Canada.  Most humans would notice the difference.

Humans and natural selection are insane in different stable complicated ways.

Comments (37)

Sort By: Old
Comment author: michael_vassar3 13 February 2009 03:23:59PM 3 points [-]

I question the evidential value of the statement below. It seems to me that it argues against evolutionary fine tuning.

" Similarly, the graph that correlates to parental grief is for the future reproductive potential of a child that has survived to a given age, and not the sunk cost of raising the child which has survived to that age. (Could we get an even higher correlation if we tried to take into account the reproductive opportunity cost of raising a child of age X to independent maturity, while discarding all sunk costs to raise a child to age X?)"

Evolution should have set the cost to a given age as approximately equal to the expected benefit. It manifestly failed to do so in establishing an approximately equal gender ratio despite the larger cost of boys than girls... unless hunter gatherers Had/Have very unequal gender ratios (inversely proportional to the cost of children) but modern environments lead to FAR less selective abortion of boys.

When I proposed a study like this a few years ago as the sort of thing that evolutionary psychologists should do if they were to be taken seriously I pointed out that hypothetical grief over girls should show much lower variance than that over boys to reflect varied reproductive expectations which should be predictable by fairly early childhood.

I'm also bothered by the idea that our ancestors even had a concept of "3 years from now" distinct from "5 years from now". If they didn't shouldn't their estimates be based on environmentally impacted physiological factors like age of puberty or height which would vary between Canad and the ancestral environment?

That said, this was my exemplar when I was looking for an example of an experiment that should be done in evolutionary psychology that could boost its credibility. Updating on both its credibility and on the ability of the scientific community to integrate data. Common sense does NOT, IMHO, say that parents would be more unhappy by the death of a 12-year-old than that of an 8-year-old.

Comment author: Eliezer_Yudkowsky 13 February 2009 03:32:16PM 3 points [-]

Evolution should have set the cost to a given age as approximately equal to the expected benefit.

Don't see how this follows. At all.

Comment author: Fenn 13 February 2009 03:55:42PM 1 point [-]

Man, I missed posts like these.

Comment author: David7 13 February 2009 04:14:02PM 5 points [-]

The most obvious inelegance of this study, as described, is that it was conducted by asking human adults to imagine parental grief, rather than asking real parents with children of particular ages.

But why would evolution seek to fine tune *actual* grief? Evolution requires a difference in rates of reproduction between differing phenotypes. I don't imagine there would be any difference in reproductive rates between those parents who happened to experience no grief at the loss of a child and those parents who were grief-stricken.

It's in fact the *imagined* grief at the loss of a child that causes the parent to protect the child, or to be more risk-averse on the child's behalf. Their imagined grief is what causes the differential in gene-survival rates, and so imagined grief is what we should expect to correlate to the future reproductive potential of the child.

Of course this would be a moot point if people were perfect predictors of their future mental states, but we already that to be false.

An afterthought: Wouldn't it be nice if we could have imagined grief, but not actual grief? I guess evolution couldn't figure out how to make that happen.

Comment author: Cyan2 13 February 2009 04:36:43PM 3 points [-]

David, the inelegance is that the study asked adults in general to imagine parental grief rather than asking parents in particular. (Your correct observations about imagined versus actual grief were already set forth in the post.)

Comment author: Benja_Fallenstein 13 February 2009 04:46:38PM 2 points [-]

Might we get an even higher correlation if we tried to take into account the reproductive opportunity cost of raising a child of age X to independent maturity, while discarding all sunk costs to raise a child to age X?

I haven't done the math, but my intuition says that upon observing the highest! correlation! ever!, surely our subjective probability must go towards a high true underlying correlation and having picked a sample with a particularly high correlation? (Conditioning on the paper not being wrong due to human error or fake, of course -- I don't suspect that particularly, but surely our subjective probability of that must go up too upon seeing the !!!.) If this is correct, it seems that we should expect to see a lower correlation for the modified design, even if the underlying effect is actually stronger.

(If I'm making a thinko somewhere there, please do tell... I hope to Know My Stuff about statistics someday, but I'm just not there yet :))

Do note that the correlation is, IIUC, between the mean Canadian rating for a given age and the mean reproductive value of female !Kung of a given age, meaning that "if the correlations were tested, the degrees of freedom would be (the number of ages) - 2 = 8, not (the number of subjects - 2) as is usually the case when testing correlations for significance", so IIUC, we expect a large influence of random variation in the sample. (The authors don't actually provide p-values for the correlations.) That's not surprising, really; if the highest! correlation! ever! came from an experiment that did not allow for significant influence of random effects (because of really large sample size, say), that should make us suspicious, right? (Because if there were real effects that large, there should be other people investigating similarly large effects with statistically weaker methods, and thus occasionally getting even more extreme results?)

Comment author: Sideways2 13 February 2009 04:49:23PM 1 point [-]

People who say that evolutionary psychology hasn't made any advance predictions are (ironically) mere victims of "no one knows what science doesn't know" syndrome.

I hope this isn't pointed at me. When I wrote,

I've never seen a researcher make a prediction based on EP and then verify it via testing. Of course, that doesn't mean it hasn't happened...

I was pretty careful to make that a statement of my own probable ignorance instead of an assertion of fact.

Reading the summary and the linked abstract left me with a few questions remaining. I'd like to know what the !Kung grief-curve looks like, for example. And the reproductive-potential curves of a few other hunter-gatherer tribes wouldn't hurt, either. I find it a bit fishy that Crawford et al. found a .92 correlation with the very first curve they compared their results to, and then didn't make comparisons to any others. Maybe I'll drop by my university library on Monday and see if I can dig up the full study.

Comment author: Eliezer_Yudkowsky 13 February 2009 04:54:02PM 1 point [-]

Benja: I'd expect to see a lower correlation on the replication, and then possibly a higher correlation than that on a modification with further reproductive opportunity costs to maturity taken into account. Yes, given prior suspicion, we should suspect that a replication would show lower correlation but still high correlation.

It's also worth noting that the study broke down male and female raters and male and female children before adding it all up, and that the correlations for each subcategory were also high (eighties and nineties).

Sideways, it's pointed at a whole 'lotta people, but I should also note that I suspect you were running into an imaginability bias ("I can't see how you would verify that") rather than a search-with-no-papers-found observation.

Comment author: Benja_Fallenstein 13 February 2009 05:43:31PM 0 points [-]

Eliezer, right, thanks. And I hadn't noticed about the correlations of the subcategories...

Comment author: michael_vassar3 13 February 2009 06:01:39PM 2 points [-]

My mistake. It should only set marginal costs equal to marginal benefits for each stage in development. Cost and benefit should only be about equal at birth.

Comment author: Garrett 13 February 2009 08:13:27PM 0 points [-]

"When this curve was compared with a curve showing changes in reproductive potential over the life cycle (a pattern calculated from Canadian demographic data), the correlation was fairly strong. But much stronger - nearly perfect, in fact - was the correlation between the grief curves of these modern Canadians and the reproductive-potential curve of a hunter-gatherer people, the !Kung of Africa."

Can someone clarify this? What is this? At first I thought that it'd be the expected number of kids one would have over the rest of their life, but I don't see how that could go ever go up.

Comment author: David7 13 February 2009 08:32:39PM 0 points [-]

Wow, I sure put my foot in my mouth there. Remind me to have coffee *before* posting. :)

Comment author: Anonymous48 13 February 2009 08:42:35PM 3 points [-]

At first I thought that it'd be the expected number of kids one would have over the rest of their life, but I don't see how that could go ever go up.

An adult can start having kids right now, whereas infant has to survive to adulthood first.

Comment author: Robin_Hanson2 13 February 2009 08:58:58PM 0 points [-]

The parental grief is not even subconsciously about reproductive value - otherwise it would update for Canadian reproductive value instead of !Kung reproductive value. ... Parents do not care about children for the sake of their reproductive contribution. Parents care about children for their own sake.

This just doesn't follow. Just because there is one feature that isn't taken into account and updated optimally in grief intensity doesn't imply that nothing else is taken into account but "the childrens' own sake", whatever that means.

Comment author: Eliezer_Yudkowsky 13 February 2009 09:15:51PM 1 point [-]

Robin, you and I are now talking strictly about cognition, right? So parents may indeed find their love responsive to various features of their children that they might, perhaps, not realize that they're taking into account. It wouldn't be surprising to find them lavishing more attention on children who seem to have better chances "in life", or feeling less grief for the death of a child already sick. But anyone suggesting an explicit, cognitive, represented ulterior motive for quote reproductive value unquote - conscious or subconscious - would seem argued-against by this experiment; if this experiment doesn't argue it, what does?

Comment author: Robin_Hanson2 13 February 2009 09:24:23PM 0 points [-]

Eliezer, our choices aren't between only the two polar opposites of only caring for the children's "own sake" vs. caring smartly for their reproductive value. Yes, the fact that our grief has not update for modern fertility patterns rejects one of those poles, but that does not imply the other pole.

Comment author: Aron 13 February 2009 09:39:08PM 0 points [-]

Why does the curve descend pre-adolescence? Doesn't an average 18 year old have higher long-term reproductive potential than an 8 year old?

Comment author: Aron 13 February 2009 10:05:25PM 0 points [-]

to spare anyone the effort: I presume it's because they begin having children, and only future children are relevant.

Comment author: cifey 12 July 2010 05:32:49AM 0 points [-]

One aspect, when the kids reach reproductive age they are now at least partially competition, so the parent child relationship can take on more of a sibling flavor. So loss of an 8 yr old would could cause more grief than a 18 yr old.

Comment author: Eliezer_Yudkowsky 13 February 2009 10:47:26PM 1 point [-]

Robin, I wasn't arguing for the other pole.

Comment author: Benja_Fallenstein 14 February 2009 12:05:01AM 3 points [-]

Garrett, since Anonymous reply was a little implicit, the point is that infants have a larger chance of dying before reproducing than young adults, so expected number of future offspring increases during childhood (when at each point counting only non-deceased children).

Aron, almost; it's because they get older, and only future children are relevant. Whether they've had children won't change the value except insofar it changes the chance for future children.

Me: ...so IIUC, we expect a large influence of random variation in the sample.

Bzzzt! Wrong.

Upon more careful reading and thinking, what I understand the authors to be doing is this. They ask 436 Canadian subjects to imagine that two sons or two daughters of different specified ages died in a car accident, and ask which child the subject thinks the parent would feel more grief for. They then use the Thurstone scaling procedure to obtain a grief score for each age (1 day; 1, 2, 6, 10, 13, 17, 20, 30, 50 years).

They say that the procedure gives highly replicable results, and they have that large sample size, so no big sampling effects expected here.

They then correlate this data with reproduction value data for the same ages for the !Kung, which they got from: Howell, N. Demography of the Dobe !Kung, New York: Academic Press, 1979. This is not a random sample, it's for the whole population, so no sampling effects there.

So replication with the same populations should give a very similar result. My original argument still applies, in that the high correlation may in part be due to the choice of populations, but I was completely wrong in expecting sampling effects to play a role.

Also, I realize now that I can't really judge how extreme the correlation is (though I'll happily defer to those who say it is very large): it's too different from the usual kind of correlation in Psychology for my fledgling feeling for correlation values to apply. The usual kind of study looks at two values for each experimental subject (e.g. IQ vs. rating of looks) where this study looks at two values (Canadian ratings and !Kung reproductive value) for each of the ten age groups. In the usual kind of study, correlations >0.9 are suspiciously high, because, AFAIR, if you administer the same psychological instrument to the same subjects twice, a good correlation between the two tests is ~0.8, which means the noise from testing is just too large to get you a correlation >0.9. This obviously doesn't apply to the present study's design.

Comment author: Douglas_Knight3 14 February 2009 06:28:31AM 0 points [-]

michael vassar: varied reproductive expectations which should be predictable by fairly early childhood.

What do you make of the claim that boys are good for marriages?

It fits if you assume that the low variance of the daughter's fitness makes it less responsive to the father's presence. If the son's fitness is predictable early, this should be reflected in modern divorces, though I don't see offhand how to test it.

Comment author: michael_vassar3 14 February 2009 07:55:59AM 0 points [-]

The need for paternal resources for boys seems likely to be a motive. So is the greater ease of recognizing paternal resemblance among boys. Finally, producing boys is a weak signal of fitness by the mother.

Comment author: Tomasz_Wegrzanowski 14 February 2009 10:27:14AM 0 points [-]

>0.9 correlations in psychological studies indicate either a massive bias in study design or some new deep truth about human nature that everybody missed up to this point, in either case - get this damn experiment reproduced asap!

Comment author: Tim_Tyler 14 February 2009 11:19:05AM 1 point [-]

Re: Parents do not care about children for the sake of their reproductive contribution. Parents care about children for their own sake [...]

Except where paternity suits are involved, presumably.

Comment author: Kaj_Sotala 14 February 2009 12:24:19PM 2 points [-]

Michael Vassar: I'm also bothered by the idea that our ancestors even had a concept of "3 years from now" distinct from "5 years from now".

Humans (as well as animals) do seem to have an instinctive, analog system for comprehending magnitude, one which the formal system of numbers builds on. (See e.g. Brannon 2006 for one overview.) Weber's law seems to be a consequence of the way the analog system represents numbers. I haven't looked at the data, but I wonder how the correlation works out once you take into account the inaccuracies introduced by Weber's law. Of course, even hunter-gatherers probably had rough concepts of age categories that didn't necessarily require exact representation of age.

(For those not in the know - shamelessly copying straight from the linked paper here, being too lazy to write it up in my own words - Weber's law states that the change in stimulus intensity needed for an organism to detect a change is a constant proportion of the original stimulus intensity rather than a constant amount. For example, if an increment of 2 pounds is needed to detect a change in a 10 pound weight, then an increment of 4 pounds would be needed to detect a change in a 20 pound weight.)

Comment author: michael_vassar3 14 February 2009 07:10:22PM 1 point [-]

Weber's law applies to perceptions. You can't really perceive time on a span of years. I'm pretty certain that human's can't intuitively distinguish 18 years from 20. My post asserted that people should use rough concepts of age categories but that those categories shouldn't involve representing age. Those categories should also not correspond precisely to our categories due to improvements in nutrition and disease burden, e.g. we go through puberty earlier, grow taller, etc.

Comment author: daginikkafa 15 February 2009 04:44:21AM -1 points [-]

"The parental grief is not even subconsciously about reproductive value - otherwise it would update for Canadian reproductive value instead of !Kung reproductive value."

Or maybe the evolution did not have sufficient time to update. Most of the change in the reproductive value occurred pretty recently.

Comment author: Tim_Tyler 15 February 2009 11:52:18AM -3 points [-]

Re: The parental grief is not even subconsciously about reproductive value - otherwise it would update for Canadian reproductive value instead of !Kung reproductive value.

I think that a better way to put this would be to say that the Canadian humans miscalculate reproductive value - using subconscious math more appropriate for bushmen.

If you want to look at the the importance of reproductive value represented by children to humans, the most obvious studies to look at are the ones that deal with adopted kids - comparing them with more typical ones. For example look at the statistics about how much such kids get beaten, suffer from child abuse, die or commit suicide.

Comment author: Waldheri 15 February 2009 09:21:33PM 1 point [-]

I wonder... How does one measure grief?

Comment author: HH2 15 February 2009 10:03:16PM 0 points [-]

"I wonder... How does one measure grief?"

Posted by: Waldheri | February 15, 2009 at 04:21 PM

Using as a proxy the length of time a person plays the song "Everybody Hurts" on repeat.

Joking aside, I imagine the scale of grief doesn't matter as much as relative values: would you be sadder if X happened or if Y happened? I suppose it could be monetized somehow ("I would pay 5 to avoid X-sadness but 8 to avoid Y-sadness.") but I doubt that would be really accurate except to show relative feelings of grief - in an experimental setting, most people would highball the amounts, but the rankings of what's sadder than what would probably still be accurate.

Comment author: Bruce_K_britton 16 February 2009 03:15:26AM 1 point [-]

This is not an experiment at all, it is simply a correlational study, and the problems with using correlations to try to establish causal links are well known. It is hard enough to establish causation with an experimental design, in which there are at least two groups of subjects who are equal to start with, with one group exposed to some putatively causal treatment and the other not, and the result measured in terms of the effect of the treatment. All you have here is a just-so story, like Kipling's, and yet you have all these commenters buying it. The reason is that stories are very persuasive. But that doesn't mean they are true.

Comment author: Waldheri 16 February 2009 09:55:02AM 0 points [-]

Thanks for the response, HH.

I partly agree with Bruce K britton - surely one can find a curve that corresponds with the results from this grief study. It may very well coincide with a curve describing the relationship between the age and production of enzyme X in bacterium Y.

The question is: Why did the researchers decide to compare it to the reproductive potential curve? Were there other clues that suggested a relationship between the two?

Comment author: Bruce_K._Britton 16 February 2009 07:29:57PM 1 point [-]

Researchers planning any sort of correlational study should post their predictions in advance on something like a prediction market setting, and correlational studies should only be publishable if they can document that the predictions were made publicly in advance. I'm talking here about correlational studies that make any sort of causal claim. Anybody who has done correlational studies knows that if you correlate 20 things with each other, you can always make up a story that links the significant correlations with each other in a plausible way, and people love stories. The evolutionary psychologists have good material to make up stories, because they can talk about 'our primitive ancestors,' which is a per se interesting story. The existence of a significant correlation is the beginning of trying to establish a causal claim, not the end.

Comment author: Yvain2 16 February 2009 10:59:26PM 6 points [-]

Bruce and Waldheri, you're being unfair.

You're interpreting this as "some scientists got together one day and asked Canadians about their grief just to see what would happen, then looked for things to correlate it with, and after a bunch of tries came across some numbers involving !Kung tribesmen reproductive potential that fit pretty closely, and then came up with a shaky story about why they might be linked and published it."

I interpret it as "some evolutionary psychologists were looking for a way to confirm evolutionary psychology, predicted that grief at losing children would be linked to reproductive potential in hunter-gatherer tribes, and ran an experiment to see if this was true. They discovered that it was true, and considered their theory confirmed."

I can't *prove* my interpretation is right because the paper is gated, but in my support, I know of many studies very similar to this one that were done specifically to confirm evo psych's predictions (for example, The Adapted Mind is full of them). And most scientists don't have enough free time to go around doing studies of people's grief for no reason and then comparing it to random data sets until they get a match, nor would journals publish it if they did. And this really is exactly the sort of elegant, testable experiment a smart person would think up if ze was looking for ways to test evolutionary theory.

It's true that correlation isn't causation and so on et cetera, but if their theory really did predict the results beforehand when other theories couldn't, we owe them a higher probability for their theory upon learning of their results.

Comment author: Douglas_Knight3 17 February 2009 01:07:23AM 0 points [-]

Bruce K. Britton, Let's start with simpler things, like having people make their data and calculations available. (Or to be really simple, journals with such rules should enforce them!) Without this, you can just hide the data-mining in poorly specified protocols, not to mention fraud.

Data-mining is not that bad because it has systematic effects that an outsider can predict and account for; at least you can hope that it will wash out in the meta-analyses. This reminds me of this Robin Hanson post on how to extract experiments from the medical literature you don't trust.

Perry E. Metzger makes similar recommendations to BKB and RH replies that it's not going to happen. Actually, the medical community is moving towards things like registering studies. I worry that actions taken with a definite sense of who is the bad guy (drug companies) may make us worse off than the status quo, though I don't see any downsides to anything that is actually going forward.

Yvain, read that post.

Comment author: Bruce_K._Britton 19 February 2009 02:18:04AM 1 point [-]

Yes, making data and calculations available would help to check results.

Data-mining and story-telling are only misleading when the results are presented as evidence for causal links, which has been so badly misleading in so many cases in the history of science that it would be very helpful to regulate strongly the practice of drawing conclusions about causal links from correlational data. Registering predictions before studies are done would be very helpful in evaluating claims about causal links derived from correlational data.

If no predictions are registered in advance, everybody would know the results were from an exploratory study, which is fine. What we want to avoid is allowing people to do exploratory studies, but present them as if they were hypothesis testing studies.