You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Incorrect comments on Case Study: Testing Confirmation Bias - Less Wrong Discussion

32 Post author: gwern 02 May 2012 02:03PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (61)

You are viewing a single comment's thread. Show more comments above.

Comment author: Incorrect 02 May 2012 07:23:51PM *  2 points [-]

Why is it ethically difficult? (I don't particularly care whether it passes a university ethics review, I base whether I consider it ethical on my emotional reaction to it and emotional reaction to its consequences)

Comment author: TimS 03 May 2012 12:55:25AM 0 points [-]

There is some ethical difficulty in running a study (and performing an intervention) on subjects who don't even know that they are in a study. In the application for a research position, people thought they were applying for a position, not tested for their responses to stimuli about the use of their work.

Obviously, certain kinds of results are impossible to obtain when you tell the subjects that they are part of an experiment. The value of the results might justify the deception, particularly in cases like these two studies in which the participants did not suffer any significant harm. But it is unrealistic to pretend that no deception occurred, or that deception is not a flag for potential ethical difficulty.

Comment author: [deleted] 02 May 2012 07:27:02PM *  0 points [-]

Section 8.02, for starts. And yes, 8.05 and 8.07 provides an exception, but it's debatable whether it applies here, where harm was actually done to Seth Roberts' reputation. It's less debatable that it applies to the researcher study, where apparently no harm was done.

Comment author: gwern 02 May 2012 08:30:23PM 2 points [-]

Arguably, the researchers were harmed inasmuch as they were induced to apply more effort/time than they otherwise would have.

Comment author: TimS 03 May 2012 12:56:26AM 0 points [-]

Putting in more work to get another interesting experimental result is a harm to the researcher? On what planet?

Comment author: gwern 03 May 2012 01:21:24AM *  2 points [-]

Well, presumably if working harder on their submission was their utility-maximizing choice, they would have done so already sans experimental manipulation; if any more quality time was used up, it probably came at the expense of some other activity...

Comment author: TimS 03 May 2012 01:55:25AM 0 points [-]

It looks like I badly misunderstood your comment. When you wrote "the researchers," I thought that was a coy way of referring to yourself in reference to the two experimental results of which I questioned the ethics.

I'm not arguing for the optimality of compliance with an IRB or other "ethical" guidelines - I'm doubtful they do a reasonable job of creating morally optimal research protocols, and they clearly prevent the discovery of certain interesting or useful results - like your results from these posts that relied on deception. And that doesn't even account for the compliance costs that I now realize was the point of your comment. Oops