jkaufman comments on Results of a One-Year Longitudinal Study of CFAR Alumni - Less Wrong

33 Post author: Unnamed 12 December 2015 04:39AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (35)
Tags: survey cfar

Comments (35)

You are viewing a single comment's thread. Show more comments above.

Comment author: AstraSequi 12 December 2015 09:28:14PM *  7 points [-]

The primary weakness of longitudinal studies, compared with studies that include a control group

Longitudinal studies can and should include control groups. The difference with RCTs is that the control group is not randomized. Instead, you select from a population which is as similar as possible to the treatment group, so an example is a group of people who were interested but couldn't attend because of scheduling conflicts. There is also the option of a placebo substitute like sending them generic self-help tips.

ETA: "Longitudinal" is also ambiguous here. It means that data were collected over time, and could mean one of several study types (RCTs are also longitudinal, by some definitions). I think you want to call this a cohort study, except without controls this is more like two different cross-sectional studies from the same population.

Comment author: jkaufman 14 December 2015 08:11:25PM *  4 points [-]

Instead, you select from a population which is as similar as possible to the treatment group

They did this with an earlier batch (I was part of that control group) and they haven't reported that data. I found this disappointing, and it makes me trust this round of data less.

On Sunday, Sep 8, 2013 Dan at CFAR wrote:

Last year, you took part in the first round of the Center for Applied Rationality's study on the benefits of learning rationality skills. As we explained then, there are two stages to the survey process: first an initial set of surveys in summer/fall 2012 (an online Rationality Survey for you to fill out about yourself, and a Friend Survey for your friends to fill out about you), and then a followup set of surveys one year later in 2013 when you (and your friends) would complete the surveys again so that we could see what has changed.

Comment author: Unnamed 16 December 2015 03:14:01PM 5 points [-]

You're right, we should've posted the results on our previous study. I'll put those numbers together in a comprehensible format and then I'll have them posted soon.

The brief explanation of why we didn't take the time to write them up earlier is that the study was underpowered and we thought that the results weren't that informative. In retrospect, that decision was a mistake.

I've put a list of the workshop surveys that we've done in a separate comment.

Powered by Reddit Powered by Reddit