Jonathan_Graehl comments on She Blinded Me With Science - Less Wrong

13 Post author: Jonathan_Graehl 04 August 2009 07:10PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (38)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 04 August 2009 10:04:13PM 10 points [-]

The obvious solution, IMO, is to have journals approve study designs for publication in advance, including all statistical tools to be used; and then you do the study and run the preselected analysis and publish the results, regardless of whether positive or negative.

But just like many other obvious improvements we can all think of to the process of science, this one will not be carried out.

parapsychology ("the control group for science")

Did you get that off me? I was planning a post on it at some point or another.

Comment author: Jonathan_Graehl 04 August 2009 10:52:57PM 0 points [-]

have journals approve study designs for publication in advance, including all statistical tools to be used; and then you do the study and run the preselected analysis and publish the results, regardless of whether positive or negative

Brilliant.

Maybe a notary service for such plans would become popular from the ground up. Of course, to get voluntary adoption, you'd have to implement a guarantee of secrecy for a desired time period (even though the interests of science would be best served by early publicity, those scientists want their priority).

Let's see, just the right protocol for signing/encrypting, and ... never mind, it will never be used until some high status scientists want to show off ;)