Follow-up to: Announcing the Quantified Health Prize

I am happy to announce that Scott Siskind, better known on Less Wrong as Yvain, has won the first Quantified Health Prize, and Kevin Fischer has been awarded second place. There were exactly five entries, so the remaining prizes will go to Steven Kaas, Kevin Keith and Michael Buck Shlegeris.

The full announcement can be found here until the second contest is announced, and is reproduced below the fold. 

While we had hoped to receive more than five entries, I feel strongly that we still got our money’s worth and more. Scott Siskind and Kevin Fischer in particular put in a lot of work, and provided largely distinct sets of insight into the question. In general, it is clear that much time was usefully spent, and all five entries had something unique to contribute to the problem.

We consider the first contest a success, and intend to announce a second contest in the next few weeks that will feature multiple questions and a much larger prize pool.

Discussion of all five entries follows:Place ($500):

5th Place ($500): Full report

Steven Kaas makes a well-reasoned argument for selenium supplementation. That obviously wasn't a complete entry. It's very possible this was a strategic decision in the hopes there would be less than five other entries, and if so it was a smart gamble that paid off. I sympathize with his statements on the difficulty of making good decisions in this space.

4th Place ($500):

4th Place ($500): Full report

Kevin Keeth’s Recommendation List is as follows: “No quantified recommendations were developed. See ‘Report Body’ for excruciating confession of abject failure.” A failure that is admitted to and analyzed with such honesty is valuable, and I’m glad that Kevin submitted an entry rather than giving up, even though he considered his entry invalid and failure is still failure. Many of the concerns raised in his explanation are no doubt valid concerns. I do think it is worth noting that a Bayesian approach is not at a loss when the data is threadbare, and the probabilistic consequences of actions are highly uncertain. Indeed, this is where a Bayesian approach is most vital, as other methods are forced to throw up their hands. Despite the protests, Kevin does provide strong cases against supplementation of a number of trace minerals that were left unconsidered by other entries, which is good confirmation to have.

3rd Place ($500):

3rd Place ($500): Full report

Michael Buck Shlegeris chose to consider only five minerals, but made reasonable choices of which five to include. None of the recommendations made on those five seem unreasonable, but the reasoning leading to them is unsound. This starts with the decision to exclude studies with less than a thousand participants. While larger sample sizes are obviously better (all else being equal), larger studies also tend to be retrospective, longitudinal monitoring studies and meta-analyses. The conclusions in each section are not justified by the sources cited, and the RDI (while a fine starting point) is leaned on too heavily. There is no cost/benefit analysis, nor are the recommendations quantified. This is a serious entry, but one that falls short.

2nd Place ($1000):

2nd Place ($1000): Full report

Kevin Fischer provides a treasure trove of information, teasing out many fine details that the other entries missed, and presented advocacy of an alternate approach that treats supplementation as a last resort far inferior to dietary modifications. Many concerns were raised about method of intake, ratios of minerals, absorption, and the various forms of each mineral. This is impressive work. There is much here that we will need to address seriously in the future, and we’re proud to have added Kevin Fischer to our research team; he has already been of great help, and we will no doubt revisit these questions.

Unfortunately, this entry falls short in several important ways. An important quote from the paper:

"“Eat food high in nutrients” represents something like the null hypothesis on nutrition - human beings were eating food for millions of years before extracting individual constituents was even possible. “Take supplements” is the alternative hypothesis.

This is an explicitly frequentist, and also Romantic, approach to the issue. Supplementation can go wrong, but so can whole foods, and there’s no reason to presume that what we did, or are currently doing with them, is ideal. Supplementation is considered only as a last resort, after radical dietary interventions have “failed,” and no numbers or targets for it are given. No cost-benefit analysis is done on either supplementation or on the main recommendations.

Winner ($5000): Scott Siskind (Yvain)

Winner: Scott Siskind / Yvain ($5000): Full report

Scott Siskind’s entry was not perfect, but it did a lot of things right. An explicit cost/benefit analysis was given, which was very important. The explanations of the origins of the RDAs were excellent, and overall the analysis of various minerals was strong, although some factors found by Kevin were missed. Two of the analyses raised concerns worth noting: potassium and sodium.

On sodium, the concern was that the analysis treated the case as clear cut when it was not; there have been challenges to salt being bad, as referenced last year by Robin Hanson, and the anti-salt studies are making the two-stage argument that blood pressure causes risks and salt raises blood pressure, rather than looking at mortality. However, the conclusions here are still reasonable, especially for ordinary Americans regularly eating super-stimulus foods loaded with the stuff.