= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on 2012 Survey Results - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (640)
By the way, has anyone figured out how to load the CSV in R?
read.tablechokes with problems like "Error in read.table("for_public.csv", header = TRUE, quote = "", sep = ",") : more columns than column names";dos2unixdoesn't help, and loading it up in OpenOffice, it looks fine but re-exporting as CSV leads to the same darn errors. Mucking about, I can get it to load if I delete everything but the first entry and then the last 4 commas but this solution doesn't work for any additional entries!This worked "dat <- read.csv('http://raikoth.net/Stuff/LessWrong/for_public.csv')"
That works, thanks.
Another R issue. How do I convert the scores from IQTest into real numbers? as.numeric seems to do strange things.
use "as.numeric(as.character(dat$IQTest))"
The IQtest data is stored as factor. A factor variable has a set of levels, numbered 1,2,3,... that are the variable can possibly take on and labels for those factors. as.numeric(X) returns the level numbers of X. as.character returns the labels of X. In the case that the labels are actually numbers (usually integers that R is interpreting as character labels for some reason), as.numeric(as.character(X)) will return the numeric values that R is interpreting as labels.
EDIT:
In this case, when no value for IQtest was reported, it was stored as " " instead of "", which made R think the variable contained character data which R defaults to treating as factors. The " "'s should all be NA's once it's converted properly.
"iqt <- as.numeric(dat$IQTest)" The already numeric IQ is in dat$IQ, iqt is only the suspect, online IQtest.
A serious design flaw in S/R means that your code fails badly and silently. The man page suggests that you try as.numeric(factor(5:10)).
Matt Simpson's code is correct. The man page suggests slightly faster as.numeric(levels(dat$IQTest))[dat$IQTest]. That example also shows why they chose the the coercion that they did.
If you really did quote="", then you don't have any quote character and it won't work. But that's probably some kind of markdown error. The default in read.table is to allow both double and single quotes, while the default in read.csv is to only allow single quotes; I find that if I change your argument to quote="\"" to allow only double quotes, then it reads it with no errors. Another difference between read.table and read.csv is that read.table defaults to allowing # comments, which mangles one of the lines. This can be fixed with comment.char="", at which point read.csv and read.table produce the same result.