evand comments on Fixing Moral Hazards In Business Science - Less Wrong

33 Post author: DavidLS 18 October 2014 09:10PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (96)

You are viewing a single comment's thread. Show more comments above.

Comment author: evand 20 October 2014 12:37:17AM 2 points [-]

I sincerely hope that study plan would not pass muster. Doesn't there need to be a more reasonable placebo?

In general, who will review proposed studies for things like suitable placebo decisions?

Comment author: DavidLS 20 October 2014 01:55:45AM 4 points [-]

Can you provide an example of what you'd like to see pass muster?

Comment author: evand 03 November 2014 03:41:20AM 0 points [-]

Roughly speaking: "Act normal" vs "Use Beeminder" vs "some alternative intervention". Basically, I expect to see "do something different" produce results, at least for a little while, for almost any value of "something different". Literally anything at all that didn't make it clear to the placebo group that they were the placebo group. Maybe some non-Beeminder exercise and intake tracking. Maybe a prescribed simple exercise routine + non-Beeminder tracking.

Comment author: DavidLS 20 October 2014 01:55:02AM 2 points [-]

I'm glad you're here. My background is in backend web software, and stats once the data has been collected. I read "Measuring the Weight of Smoke" in college, but that's not really a sufficient background to design the general protocol. That's a lot of my motivation behind posting this to LW - there seem to be protocol experts here, with great critiques of the existing ones.

My hope is we can create a "getting started testing" document that gets honest companies on the right track. Searching around the web I'm finding things like this rather than serious guides to proper placebo creation.

In general, who will review proposed studies for things like suitable placebo decisions?

I'm hoping either registered statistical consultants or grad students. Hopefully this can be streamlined by a good introductory guide.