Will_Newsome comments on (Virtual) Employment Open Thread - Less Wrong

35 Post author: Will_Newsome 23 September 2010 04:25AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (276)

You are viewing a single comment's thread. Show more comments above.

Comment author: Will_Newsome 25 October 2010 03:48:41AM 2 points [-]

Yeah, I've done many of those. I took the SAT when I was 12. I've taken a few probably-inaccurate online IQ tests. I've done a few cognitive testing suites at SIAI. I'm in pretty good shape. In general though, there are better frameworks for cognitive testing. It's probable that one could make a neat suite out of PEBL, which is free and very customizable. Fluid g seems over-emphasized. The limiting factor for most rationalists tends to be strong metadispositions for thought, reflection, and drive.

Comment author: topynate 25 October 2010 04:45:26AM 0 points [-]

I've done a few cognitive testing suites at SIAI.

They have those?

Comment author: Will_Newsome 25 October 2010 08:34:00AM 0 points [-]

They're ad hoc, we've used one for a dual n-back study which ended up yielding insufficient data.

Comment author: gwern 27 October 2010 10:21:08PM *  2 points [-]

Any chance you could write up that study? I don't believe I have seen any SIAI-related DNB study; certainly it's not in my FAQ.

(Remember kids: only you can fight publication bias!)

Comment author: Will_Newsome 28 October 2010 04:10:37AM 0 points [-]

We didn't study long enough to get any statistically significant data. Like, not even close. And I think sending off the data (even without names attached) would sorta breach an implicit privacy agreement among those who took part in the tests.

Comment author: gwern 28 October 2010 02:24:37PM 0 points [-]

Jaeggi 2008 didn't necessarily study very long either, some around/less than a week.

(I wouldn't be asking this question, by the way, if you had written more concretely and said something like 'We only studied for 2 days, not long enough to get any statistically significant data'.)

Comment author: JoshuaZ 28 October 2010 04:13:24AM 0 points [-]

Hmm, most people would be ok with that sort of data being sent out in an anonymized form. I'm surprised that you didn't suggest that before hand. Is there any chance you can contact the people in question and get their permission to release the anonymized data?

Comment author: Will_Newsome 28 October 2010 04:24:21AM 0 points [-]

Is there any chance you can contact the people in question and get their permission to release the anonymized data?

We could, but really, there's no information there, no matter how much Bayes magic you use. It's noise. If the data was at all significant then we'd send it out, of course. We might actually have gotten anonymized disclosure agreement from everyone; I don't remember. But it didn't end up mattering.