gwern comments on 2012 Survey Results - Less Wrong

80 Post author: Yvain 07 December 2012 09:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (640)

Sort By: Controversial

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 02 December 2012 12:56:26AM *  2 points [-]

To add a linear model (for those unfamiliar, see my HPMoR examples) which will really just recapitulate the simple averages calculation:

R> lw2009 <- read.csv("lw-2009.csv")
R> lw2011 <- read.csv("lw-2011.csv")
R> lw2012 <- read.csv("lw-2012.csv")
R>
R> # lwi2009 <- lw2009$IQ[!is.na(lw2009$IQ)]
R> # hand-cleaned:
R> lwi2009 <- c(120,125,128,129,130,130,130,130,130,130,130,130,130,131,132,132,133,134,136,138,138,139,139,140,
R> 140,140,140,140,140,140,140,140,141,142,144,145,145,145,148,148,150,150,150,150,152,154,154,
R> 155,155,155,156,158,158,160,160,160,160,162,163,164,165,166,170,171,173,180)
R> lwi2011 <- lw2011$IQ[!is.na(lw2011$IQ)]
R> lwi2012 <- lw2012$IQ[!is.na(lw2012$IQ)]
R>
R> xs <- c(rep(as.Date("2009-03-01"), length(lwi2009)), rep(as.Date("2011-11-01"), length(lwi2011)), rep(as.Date("2012-11-01"), length(lwi2012)))
R> ys <- c(lwi2009, lwi2011, lwi2012)
R> model <- lm(ys ~ xs)
R> summary(model)
Call:
lm(formula = ys ~ xs)
Residuals:
Min 1Q Median 3Q Max
-38.29 -8.29 -0.29 6.73 63.81
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 219.49064 19.42751 11.30 < 2e-16
xs -0.00519 0.00126 -4.11 4.5e-05
Residual standard error: 12.9 on 741 degrees of freedom
Multiple R-squared: 0.0222, Adjusted R-squared: 0.0209
F-statistic: 16.9 on 1 and 741 DF, p-value: 4.48e-05

<code>plot(xs,ys); abline(model)</code>

Comment author: satt 02 December 2012 10:21:25AM 4 points [-]

Note that Epiphany dates the 2009 survey to around March, while the other two surveys happened around November, so inputting the survey dates just as years lowballs the time gap between the first & second surveys. Your linear trend'll be a bit exaggerated.

Comment author: gwern 02 December 2012 06:55:26PM 4 points [-]

I've fixed it as appropriate.

Your linear trend'll be a bit exaggerated.

Before, the slope per year was -2.24 (minus 2.25 points a year), now the slope spits out as -0.00519 but if I'm understanding my changes right, the unit has switched from per year to per day and 365.25 times -0.005 IQ points per day is -1.896 per year.

2.25 vs 1.9 is fairly different.