Vaniver comments on 2012 Survey Results - Less Wrong

80 Post author: Yvain 07 December 2012 09:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (640)
Tags: survey meta community

Comments (640)

Sort By: Popular
Best Popular New Controversial Top Old Leading

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 30 November 2012 05:47:35AM 5 points [-]

I am also surprised by this. I wonder about the effect of "I'm taking this survey so I don't have to go to bed / do work / etc.," but I wouldn't have expected that to be as large as the diligence effect.

Also, perhaps look at nonresponse by section? I seem to recall the C part being after the personality test, which might be having some selection effects.

Comment author: gwern 30 November 2012 04:13:11PM 1 point [-]

Also, perhaps look at nonresponse by section? I seem to recall the C part being after the personality test, which might be having some selection effects.

What do you mean? I can't compare non-response with anyone who didn't supply a C score, and there were plenty of questions to non-response on after the personality test section.

Comment author: Vaniver 30 November 2012 05:28:39PM 2 points [-]

It seems to me that other survey non-response may be uncorrelated with C once you condition on taking a long personality survey, especially if the personality survey doesn't allow nonresponse. (I seem to recall taking all of the optional surveys and considering the personality one the most boring. I don't know how much that generalizes to other people.) The first way that comes to mind to gather information for this is to compare the nonresponse of people who supplied personality scores and people who didn't, but that isn't a full test unless you can come up with another way to link the nonresponse to C.

I was thinking it might help to break down the responses by section, and seeing if nonresponse to particular sections was correlated with C, but the result could only be that some sections are anticorrelated if a few are correlated. So that probably won't get you anything.

Comment author: gwern 01 December 2012 12:21:20AM 2 points [-]

It seems to me that other survey non-response may be uncorrelated with C once you condition on taking a long personality survey, especially if the personality survey doesn't allow nonresponse. (I seem to recall taking all of the optional surveys and considering the personality one the most boring. I don't know how much that generalizes to other people.)

Why would the strong correlation go away after adding a floor? That would simply restrict the range... if that were true, we'd expect to see a cutoff for all C scores but in fact we see plenty of very low C scores being reported.

The first way that comes to mind to gather information for this is to compare the nonresponse of people who supplied personality scores and people who didn't, but that isn't a full test unless you can come up with another way to link the nonresponse to C.

Yes. You'd expect, by definition, that people who answered the personality questions would have fewer non-responses than the people who didn't... That's pretty obvious and true:

R> lwc <- subset(lw, !is.na(as.integer(as.character(BigFiveC))))
R> missing_answers1 <- apply(lwc, 1, function(x) sum(sapply(x, function(y) is.na(y) || as.character(y)==" ")))
R> lwnc <- subset(lw, is.na(as.integer(as.character(BigFiveC))))
R> missing_answers2 <- apply(lwnc, 1, function(x) sum(sapply(x, function(y) is.na(y) || as.character(y)==" ")))
R> t.test(missing_answers1, missing_answers2)
Welch Two Sample t-test
data: missing_answers1 and missing_answers2
t = -25.19, df = 806.8, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-18.77 -16.05
sample estimates:
mean of x mean of y
9.719 27.129
Powered by Reddit Powered by Reddit