You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

MaximumLiberty comments on Fixing Moral Hazards In Business Science - Less Wrong Discussion

33 Post author: DavidLS 18 October 2014 09:10PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (96)

You are viewing a single comment's thread. Show more comments above.

Comment author: MaximumLiberty 19 October 2014 08:21:18PM 2 points [-]

I think I get your meaning. You mean that the webapp itself would carry out the testing protocol. I was thinking that it would be designed by the sponsor using standardized components. I think what you are saying is that it would be more rigid than that. This would allow much more certainty in the meaning of the result. Your example of "using X resulted in average weight loss of Y compared to a control group" would be a case that could be standardized, where "average weight loss" is a configurable data element.

Max L.

Comment author: DavidLS 19 October 2014 10:37:25PM 1 point [-]

Yes. I think if we can manage it, requiring data-analysis to be pre-declared is just better. I don't think science as a whole can do this, because not all data is as cheap to produce as product testing data.

Now that I've heard your reply to question #8, I need to consider this again. Perhaps we could have some basic claims done by software, while allowing for additional claims such as "those over 50 show twice the results" to be verified by grad students. I will think about this.