gwern comments on Stuff That Makes Stuff Happen - Less Wrong

51 Post author: Eliezer_Yudkowsky 18 October 2012 10:49AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (127)

You are viewing a single comment's thread.

Comment author: gwern 18 October 2012 05:43:24PM 6 points [-]

Clearly, using 1-0-1-0-1-0 on a list of patients in alphabetical order isn't random enough... or is it?

It's not, if only because the people implementing it can guess it: a textbook I read on doing medical trials mentioned that this procedure was done in medical trials, and it led to tampering where doctors would send the patients they liked better or were sicker or whatever to the 'right' trial arm.

Comment author: thomblake 18 October 2012 06:35:35PM 2 points [-]

it led to tampering where doctors would send the patients they liked better or were sicker or whatever to the 'right' trial arm.

So they changed the person's name, or what?

Comment author: gwern 18 October 2012 06:53:44PM *  13 points [-]

Something like that. There are a lot of ways to tamper with this: participation is voluntary, of course, so if a patient would 'benefit' from being randomized to the 'right' arm, you'd encourage them to do it, while if they weren't, you'd encourage them to drop out (and maybe get the tested treatment themselves!). You'd filter the list in the first place, or use alternative names (My legal first name starts with one letter but I always go by a version of my middle name which starts with a different letter: which version does the doctor write down?). And so on.

One interesting example, from a retrospective:

It took only a few months to accumulate the required experience in the two hospitals (Reese et al. 1952). Allocation to ACTH or no ACTH was decided by drawing marbles from a jar containing an equal number of white and blue marbles: one morning, when a new infant became eligible for enrollment, I noticed that our head nurse shook the jar vigorously, turned her head away, pulled a marble out (just as she had been instructed); but because she didn't like the 'assignment', she put the marble back, shook the jar again, and pulled out the color that agreed with her bias! The importance of Bradford Hill's precaution in Britain's famous streptomycin trial to conceal the order of assignment in sealed envelopes was immediately obvious!

Comment author: thomblake 18 October 2012 06:55:47PM 1 point [-]

thanks!