Hi Yvain,
please state a definite end date next year. Filling out the survey didn't have a really high priority for me, but knowing that I had "about a month" made me put it off. Had I known that the last possible day was the 26th of November, I probably would have fit it in sometime in between other stuff.
The calibration question is an n=1 sample on one of the two important axes (those axes being who's answering, and what question they're answering). Give a question that's harder than it looks, and people will come out overconfident on average; give a question that's easier than it looks, and they'll come out underconfident on average. Getting rid of this effect requires a pool of questions, so that it'll average out.
Yep. (Or as Yvain suggests, give a question which is likely to be answered with a bias in a particular direction.)
It's not clear what you can conclude from the fact that 17% of all people who answered a single question at 50% confidence got it right, but you can't conclude from it that if you asked one of these people a hundred binary questions and they answered "yes" at 50% confidence, that person would only get 17% right. The latter is what would deserve to be called "atrocious"; I don't believe the adjective applies to the results observed in the survey.
I'm not even sure that you can draw the conclusion "not everyone in the sample is perfectly calibrated" from these results. Well, the people who were 100% sure they were wrong, and happened to be correct, are definitely not perfectly calibrated; but I'm not sure what we can say of the rest.
I previously mentioned that item non-response might be a good measure of Conscientiousness. Before doing anything fancy with non-response, I first checked that there was a correlation with the questionnaire reports. The correlation is zero:
R> lwc <- subset(lw, !is.na(as.integer(as.character(BigFiveC))))
R> missing_answers <- apply(lwc, 1, function(x) sum(sapply(x, function(y) is.na(y) || as.character(y)==" ")))
R> cor.test(as.integer(as.character(lwc$BigFiveC)), missing_answers)
Pearson's product-moment correlation
data: as.integer(as.character(lwc$BigFiveC)) and missing_answers
t = -0.0061, df = 421, p-value = 0.9952
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
-0.09564 0.09505
sample estimates:
cor
-0.0002954
# visualize to see if we made some mistake somewhere
R> plot(as.integer(as.character(lwc$BigFiveC)), missing_answers)
I am completely surprised. The results in the economics paper looked great and the rationale is very plausible. Yet... The 2 sets of data here have the right ranges, there's plenty of variation in both dimension, I'm sure I'm catching most of the item non-responses or N...
I really have no idea what went so wrong [with the question about Bayes' birth year]
Note also that in the last two surveys the mean and median answers were approximately correct, whereas this time even the first quartile answer was too late by almost a decade. So it's not just a matter of overconfidence -- there also was a systematic error. Note that Essay Towards Solving a Problem in the Doctrine of Chances was published posthumously when Bayes would have been 62; if people estimated the year it was published and assumed that he had been approximately in his thirties (as I did), that would explain half of the systematic bias.
On IQ Accuracy:
As Yvain says, "people have been pretty quick to ridicule this survey's intelligence numbers as completely useless and impossible and so on" because if they're true, it means that the average LessWronger is gifted. Yvain added a few questions to the 2012 survey, including the ACT and SAT questions and the Myers-Briggs personality type question that I requested (I'll explain why this is interesting), and that give us a few other things to check against, which has made the figures more believable. The ridicule may be an example of the "virtuous doubt" that Luke warns about in Overconfident Pessimism, so it makes sense to "consider the opposite":
The distribution of Myers-Briggs personality types on LessWrong replicates the Mensa pattern. This is remarkable since the patterns of personality types here are, in many significant ways, the exact opposite of what you'd find in the regular population. For instance, the introverted rationalists and idealists are each about 1% of the population. Here, they are the majority and it's the artisans and guardians who are relegated to 1% or less of our population.
Mensa's personality test results we...
Alternate possibility: The distribution of personality types in Mensa/LW relative to everyone else is an artifact produced by self-identified smart people trying to signal their intelligence by answering 'yes' to traits that sound like the traits they ought to have.
eg. I know that a number of the T/F questions are along the lines of "I use logic to make decisions (Y/N)", which is a no-brainer if you're trying to signal intelligence.
A hypothetical way to get around this would be to have your partner/family member/best friend next to you as you take the test, ready to call you out when your self-assessment diverges from your actual behaviour ("hold on, what about that time you decided not to go to the concert of [band you love] because you were angry about an unrelated thing?")
From the public dataset:
165 out of 549 responses without reported positive karma (30%) self-reported an IQ score; the average response was 138.44.
181 out of 518 responses with reported positive karma (34%) self-reported an IQ score; the average response was 138.25.
One of the curious features of the self-reports is how many of the IQs are divisible by 5. Among lurkers, we had 2 151s, 1 149, and 10 150s.
I think the average self-response is basically worthless, since it's only a third of responders and they're likely to be wildly optimistic.
So, what about the Raven's test? In total, 188 responders with positive karma (36%), and 164 responders without positive karma (30%) took the Raven's test, with averages of 126.9 and 124.4. Noteworthy is the new max and min- the highest scorer on the Raven's test claimed to get 150, and the three sub-100 scores were 3, 18, and 66 (of which I suspect only the last isn't a typo or error of some sort).
Only 121 users both self-reported IQ and took the Raven's test. The correlation between their mean-adjusted self-reported IQ and mean-adjusted Raven's test was an abysmal .2. Among posters with positive karma, the correlation was .45; among posters without positive karma, the correlation was -.11.
Ok I managed to dig it up!
E/I | S/N | T/F | J/P (Category)
----------------------------------------------
75/25 75/25 55/45 50/50 (Overall population)
27/73 10/90 75/25 65/35 (Mensans)
15/85 03/97 88/12 54/46 (LessWrongers) *
From the December 1993 Mensa Bulletin.
* The LessWrongers were added by me, using the same calculation method as in the comment where I test my personality type predictions and are based on the 2012 survey results.
This also explains a lot of things. People regard IQ as if it is meaningless, just a number, and they often get defensive when intellectual differences are acknowledged. I spent a lot of time doing research on adult giftedness (though I'm most interested in highly gifted+ adults) and, assuming the studies were done in a way that is useful (I've heard there are problems with this), and my personal experiences talking to gifted adults are halfway decent as representations of the gifted adult population, there are a plethora of differences that gifted adults have. For instance, in "You're Calling Who A Cult Leader?" Eliezer is annoyed with the fact that people assume that high praise is automatic evidence that a person has joined a cult. What he doesn't touch on is that there are very significant neurological differences between people in just about every way you could think of, including emotional excitability. People assume that others are like themselves, and this causes all manner of confusion. Eliezer is clearly gifted and intense and he probably experiences admiration with a higher level of emotional intensity than most. If the readers of LessWrong and Hacker Ne...
Eliezer is clearly gifted and intense and he probably experiences admiration with a higher level of emotional intensity than most. If the readers of LessWrong and Hacker News are gifted, same goes for many of them. To those who feel so strongly, excited praise may seem fairly normal. To all those who do not, it probably looks crazy.
Would you predict then that people who're not gifted are in general markedly less inclined to praise things with a high level of intensity?
This seems to me to be falsified by everyday experience. See fan reactions to Twilight, for a ready-to-hand example.
My hypothesis would simply be that different people experience emotional intensity as a reaction to different things. Thus, some think we are crazy and cultish, while also totally weird for getting excited about boring and dry things like math and rationality... while some of us think that certain people who are really interested in the lives of celebrities are crazy and shallow, while also totally weird for getting excited about boring and bad things like Twilight.
This also leads each group to think that the other doesn't get similar levels of emotional intensity, because only the group's own type of "emotional intensity" is classified as valid intensity and the other group's intensity is classified as madness, if it's recognized at all. I've certainly made the mistake of assuming that other people must live boring and uninteresting lives, simply because I didn't realize that they genuinely felt very strongly about the things that I considered boring. (Obligatory link.)
(Of course, I'm not denying there being variation in the "emotional intensity" trait in general, but I haven't seen anything to suggest that the median of this trait would be considerably different in gifted and non-gifted populations.)
But I am skeptical of these numbers. I hang out with some people who are very closely associated with the greater Less Wrong community, and a lot of them didn't know about the survey until I mentioned it to them in person. I know some people who could plausibly be described as focusing their lives around the community who just never took the survey for one reason or another. One lesson of this survey may be that the community is no longer limited to people who check Less Wrong very often, if at all. One friend didn't see the survey because she hangs out on the #lesswrong channel more than the main site. Another mostly just goes to meetups. So I think this represents only a small sample of people who could justly be considered Less Wrongers.
Yeah, this also fits my observations--I suspect that reading LW and hanging out with LW types in real life are substitute goods.
Some of the 'descriptions of LessWrong' can make for a great quote on the back of Yudkowsky's book.
Obnoxious self-serving, foolish trolling dehumanizing pseudointellectualism, aesthetically bankrupt.
;-)
Pratchett always includes a quote that calls him a "complete amateur," so there is some precedent for ostentatiously including negative reviews.
Before even reading the full details, I want to congratulate to you for the impressive amount of work. The survey period is possibly my favorite time of the year on lesswrong!
EDIT: The links for the raw csv/xls data at the bottom don't seem to work for me.
Top 100 Users' Data, aka Karma 1000+
I was thinking about the fact that there is probably a difference between active LWers versus lurkers or newbies. So I looked at the data for the Top 100 users (actually Top 107, because there was a tie). This happily coincided with the nice Schelling point of 1000 karma. (make sense, because people are likely to round to that number.) To me, this reads as "has been actively posting for at least a year".
So, some data on 1000+ karma people:
Slightly more likely to be male:
92.5% Male, 7.4% Female
NOTE: First percentage is for 1000+ users, second number is for all survey respondents
Much more likely to be polyamorous:
Prefer mono 36% v. 54%
Prefer poly 24% v. 13%
Uncertain 33% v. 30%
Other 4% v. 2%
About the same Age:
average 28.6 v. 27.8
About as likely to be single
51% v. 53%
Equally likely to be vegetarian
12%
Much more likely to use modafinil at least once per month:
15% v. 4%
About equal on intelligence tests
SAT out of 1600: 1509 v. 1486
SAT out of 2400: 2260 v. 2319
Self reported IQ: 138.5 v. 138.7
online IQ test: 127 v. 126
ACT score: ...
"Eliezer Yudkowsky personality cult."
"The new thing for people who would have been Randian Objectivists 30 years ago."
"A sinister instrument of billionaire Peter Thiel."
Nope, no one guessed whose sinister instrument this site is. Muaha.
So I suggest that we now have pretty good, pretty believable evidence that the average IQ for this site really is somewhere in the 130s, and that self-reported IQ isn't as terrible a measure as one might think.
This still suffers from selection bias - I'd imagine that people with lower IQ are more likely to leave the field blank than people with higher IQ.
This still suffers from selection bias - I'd imagine that people with lower IQ are more likely to leave the field blank than people with higher IQ.
I think this is only true if we're going to also assume that the selection bias is operating on ACT and SAT scores. But we know they correlate with IQ, and quite a few respondents included ACT/SAT1600/SAT2400 data while they didn't include the IQ; so all we have to do is take for each standardized test the subset of people with IQ scores and people without, and see if the latter have lower scores indicating lower IQs. The results seem to indicate that while there may be a small difference in means between the groups on the 3 scores, it's neither of large effect size nor statistical significance.
ACT:
R> lwa <- subset(lw, !is.na(as.integer(ACTscoreoutof36)))
R> lwiq <- subset(lwa, !is.na(as.integer(IQ)))
R> lwiqnot <- subset(lwa, is.na(as.integer(IQ)))
R> t.test(lwiq$ACTscoreoutof36, lwiqnot$ACTscoreoutof36, alternative="less")
Welch Two Sample t-test
data: lwiq$ACTscoreoutof36 and lwiqnot$ACTscoreoutof36
t = 0.5088, df = 141.9, p-value = 0.6942
alternative hypothesis: true difference in means is lOr we could just be really horrible. If we haven't even learned to avoid the one bias that we can measure super well and which is most susceptible to training, what are we even doing here?
You're fun to read. Posts explaining things and introducing terms that connect subjects and form patterns trigger reward mechanisms in the brain. This is uncorrelated to actually applying any lessons in daily life.
Two questions you might want to ask next year is "do you think it is practical and advantageous to reduce people's biases via standardized exercises?" and "Has reading LW inspired you to try and reduce your own biases?"
If we haven't even learned to avoid the one bias that we can measure super well and which is most susceptible to training, what are we even doing here?
This sounds like a job for cognitive psychology!
"Well-calibrated" should probably be improved to "well-calibrated about X"-- it's plausible that people have better and worse calibration about different subjects, and the samples in the survey only explored a tiny part of calibration space.
The 2011 survey ran 33 days and collected 1090 responses. This year's survey ran 23 days and collected 1195 responses.
Why did you close it early? That seems entirely unnecessary.
One friend didn't see the survey because she hangs out on the #lesswrong channel more than the main site.
I put a link and exhortation prominently in the #lesswrong topic from the day the survey opened to the day it closed.
M (trans f->m): 3, 0.3% / F (trans m->f): 16, 1.3%
3 vs 16 seems like quite a difference, even allowing for the small sample size. Is this consistent with the larger population?
Prefer polyamorous: 155, 13.1%...NUMBER OF CURRENT PARTNERS:... [>1 partners = 4.5%]
So ~3x more people prefer polyamory than are actually engaged in it...
Referred by HPMOR: 262, 22.1%
Impressive.
gwern.net: 5 people
Woot! And I'm not even trying or linking LW especially often.
(I am also pleased by the nicotine and modafinil results, although you dropped a number in 'Never: 76.5%')
TROLL TOLL POLICY: Disapprove: 194, 16.4% Approve: 178, 15%
So more people are against than for. Not exactly a mandate for its use.
...Are people who understand quantum mechanics are more likely to believe in Ma
So ~3x more people prefer polyamory than are actually engaged in it...
I would not describe this as an accurate conclusion. For one thing, I currently have one partner who has other partners, so I think I am unambiguously "currently engaged in polyamory" even though I would have put 1 on the survey.
For another, I think it is reasonable to say that someone who is in a relationship with exactly one other person, but is not monogamous with that person (i.e. is available to enter further relationships) is engaged in polyamory.
3 vs 16 seems like quite a difference, even allowing for the small sample size. Is this consistent with the larger population?
What struck me was not the difference in numbers of FtM and MtF, but the fact that more than ten percent of the survey population identifying as female is MtF.
Hypothesis: those directly affected by the troll policy (trolls) are more likely to have strong disapproval than those unaffected by the troll policy are to have strong approval.
Hypothesis rejected when we operationalize 'trolls' as 'low karma':
R> lwtroll <- lw[!is.na(lw$KarmaScore),]
R> lwtroll <- lwtroll[lwtroll$TrollToll=="Agree with toll" | lwtroll$TrollToll=="Disagree with toll",]
R> # disagree=3, agree=2; so:
R> # if positive correlation, higher karma associates with disagreement
R> # if negative correlation, higher karma associates with agreement
R> # we are testing hypothesis higher karma = lower score/higher agreement
R> cor.test(as.integer(lwtroll$TrollToll), lwtroll$KarmaScore, alternative="less")
Pearson's product-moment correlation
data: as.integer(lwtroll$TrollToll) and lwtroll$KarmaScore
t = 1.362, df = 315, p-value = 0.9129
alternative hypothesis: true correlation is less than 0
95 percent confidence interval:
-1.0000 0.1679
sample estimates:
cor
0.07653
R> # a log-transform of the karma scores doesn't help:
R> cor.test(as.integer(lwtroll$TrollToll), log1p(lwtroll$KarmaScore), alternIf this were anywhere but a site dedicated to rationality, I would expect trolls to self-report their karma scores much higher on a survey than they actually are, but that data is pretty staggering. I accept the rejection of the hypothesis, and withdraw my opinion insofar as it applies to this site.
After I posted my comment, I realized that 3 vs 16 might just reflect the overall gender ratio of LW: if there's no connection between that stuff and finding LW interesting (a claim which may or may not be surprising depending on your background theories and beliefs), then 3 vs 16 might be a smaller version of the larger gender sample of 120 vs 1057. The respective decimals are 0.1875 and 0.1135, which is not dramatic-looking. The statistics for whether membership differs between the two pairs:
R> M <- as.table(rbind(c(120, 1057), c(3,16)))
R> dimnames(M) <- list(status=c("c","t"), gender=c("M","F"))
R> M
gender
status M F
c 120 1057
t 3 16
R> chisq.test(M, simulate.p.value = TRUE, B = 20000000)
Pearson's Chi-squared test with simulated p-value (based on 2e+07 replicates)
data: M
X-squared = 0.6342, df = NA, p-value = 0.4346
(So it's not even close to the usual significance level. As intuitively makes sense: remove or add one person in the right category, and the ratio changes a fair bit.)
Under this theory, it seems (with low statistical confidence of course) that LW-interest is perhaps correlated with biological sex rather than gender identity, or perhaps with assigned-gender-during-childhood. Which is kind of interesting.
It's more complicated than that. Gender nonconformity in childhood is frequently punished, so a great many trans people have some very powerful incentives to suppress or constrain our interests early in life, or restrict our participation in activities for which an expressed interest earns censure or worse.
Pragmatically, gender is also performed, and there are a lot of subtle little things about it that cisgender people don't necessarily have innately either, but which are learned and transmitted culturally, many of which are the practical aspects of larger stuff (putting on makeup and making it look good is a skill, and it consists of lots of tiny subskills). Due to the aforementioned process, trans people very frequently don't get a chance to acquire those skills during the phase when their cis counterparts are learning them, or face more risks for doing so.
Finally, at least in the West: Trans medical and social access were originally predicated on jumping through an awful lot of very heteronormative hoops, and that framework still heavily influences many trans communities, particularly for older folks. This aspect is changing much faster thanks to the internet, but you still o...
After I posted my comment, I realized that 3 vs 16 might just reflect the overall gender ratio of LW
Now I feel dumb for not even noticing that. "In a group where most people were born males, why is it the case that most trans people were born males?" doesn't even seem like a question.
I'm attracted mostly to females because of instinct (I guess?) and practical issues (e.g. disdain of anal sex).
If you can't think of practical ways for two people with penises to have sex that don't involve anal, you might just need better porn.
Thank you for this public service. It seems definitely helpful for the community, and possibly helpful for historians :-)
and possibly helpful for historians :-)
I now have this mental image of future sociology grad students working on their theses by reading through every article and comment ever posted on Less Wrong, and then analyzing us.
I now have an image of those sociologists giving up on reading everything and writing scripts to do some sort of ngram or inverse-markov analysis, then mis-applying statistics to draw wrong conclusions from it. Am I cynical yet?
According to IQ Comparison Site, an SAT score of 1485/1600 corresponds to an IQ of about 144. According to Ivy West, an ACT of 33 corresponds to an SAT of 1470 (and thence to IQ of 143).
Only if you took the SAT before 1994. Here's the percentiles for SATs taken in 2012; someone who was 97th percentile would get ~760 on math and ~730 on critical reading, adding up to 1490 (leaving alone the writing section to keep it within 1600), and 97th percentile corresponds to an IQ of 128.
Here's a classic calibration chart.
An important part of the calibration chart (for people) is the frequency of times that they provide various calibrations. Looking at your table, I would focus on the large frequency between 10% and 30%.
I'll also point out that fixed windows are a pretty bad way to do elicitation. I tend to come from the calibration question from the practical side: how do we get useful probabilities out of subject-matter experts without those people being experts at calibration? Adopting those strategies seems more useful than making people experts at calibration.
Are people who understand quantum mechanics are more likely to believe in Many Worlds? We perform a t-test, checking whether one's probability of the MWI being true depends on whether or not one can solve the Schrodinger Equation. People who could solve the equation had on average a 54.3% probability of MWI, compared to 51.3% in those who could not. The p-value is 0.26; there is a 26% probability this occurs by chance. Therefore, we fail to establish that people's probability of MWI varies with understanding of quantum mechanics.
Some Bayesian analysis using the BEST MCMC library for normal two-group comparisons:
R> lw <- read.csv("lw-2012.csv")
R>
R> lwm <- subset(lw, !(" " == as.character(SchrodingerEquation)))
R> lwm <- subset(lwm, !is.na(as.integer(as.character(PManyWorlds))))
R> mwiyes <- as.integer(as.character(subset(lwm, SchrodingerEquation == "Yes")$PManyWorlds))
R> mwino <- as.integer(as.character(subset(lwm, SchrodingerEquation == "No")$PManyWorlds))
R>
R> source("BEST.R")
R> mcmcChain = BESTmcmc(mwino, mwiyes)
R> show(postInfo)
SUMMARY.INFO
PARAMETER mean When you discuss the calibration results, could you mention that the surveyors were told what constituted a correct answer? I didn't take the survey and it isn't obvious from reading this post. Also, could you include a plug for PredictionBook around there? You've included lots of other helpful plugs.
The lesson I have drawn from the survey is that free-response text fields are the devil and no one is to be trusted with them.
Yvain, I rechecked the calibration survey results, and encourage someone to recheck my recheck further:
First, these strata overlap... is 5 in 0-5 or 5-15? The N I doesn't actually match either one get either one when I recheck.
Secondly, I am not sure what program you used to calculate the statistics, but when I checked in excel, some people used percentages that got pulled as numbers less than one. I tried to clean that for these. (also removed someone who answered 150.)
Thirdly, there are 20 people in this N. You can be either 60% correct (12 correct), or 65% correct (13 correct), but 60.2% correct in this line seems weird. 85-95: 60.2% [n = 20]
Here was my attempt at recalculating those figures: N after data cleaning was 998.
0-<5: 9.1% [n = 2/22]
5-<15: 13.7% [n = 25/183]
15-<25: 9.3% [n = 21/226]
25-<35: 10% [n = 20/200]
35-<45: 11.1% [n = 10/90]
45-<55: 17.3% [n = 19/110]
55-<65: 20.8% [n = 11/53]
65-<75: 22.6% [n = 7/31]
75-<85: 36.7% [n = 11/30]
85-<95: 63.2% [n = 12/19]
95-100: 88.2% [n = 30/34]
I express low confidence in these remarks because I haven't rechecked this or gone into detail about data cleaning, but my brief take is:
1: Yes, there were some e...
In the fair coin questions, there were two people answering 49.9, one 49.9999, one 49.999999, and one 51. :-/
Other Christian: 517, 43.6% Catholic: 295, 24.9%
Now that I think about that, lumping Protestants and Orthodoxes together and keeping Catholics separate is about as bizarre as it gets.
Pandemic (bioengineered): 272, 23% Environmental collapse: 171, 14.5% Unfriendly AI: 160, 13.5% Nuclear war: 155, 13.1% Economic/Political collapse: 137, 11.6% Pandemic (natural): 99, 8.4% Nanotech: 49, 4.1% Asteroid: 43, 3.6%
This is one question where the results really suprised me. Combining natural and engendered pandemics, almost a third of respondents picked it as the top near term X risk which was almost twice as many as the next highest risk. I wonder if the x risk discussions we tend to have may be somewhat misallocated.
Note that the question on the survey was not about existential risks:
Type of Global Catastrophic Risk
Which disaster do you think is most likely to wipe out greater than 90% of humanity before the year 2100?
I answered bio-engineered pandemics, but would have answered differently for x-risks.
These are the results of the CFAR questions; I have also posted this as its own Discussion section post.
SUMMARY: The CFAR questions were all adapted from the heuristics and biases literature, based on five different cognitive biases or reasoning errors. LWers, on the whole, showed less bias than is typical in the published research (on all 4 questions where this was testable), but did show clear evidence of bias on 2-3 of those 4 questions. Further, those with closer ties to the LW community (e.g., those who had read more of the sequences) showed signifi...
MORE DETAILED RESULTS
There were 5 questions related to strength of membership in the LW community which I standardized and combined into a single composite measure of LW exposure (LW use, sequence reading, time in community, karma, meetup attendance); this was the main predictor variable I used (time per day on LW also seems related, but I found out while analyzing last year's survey that it doesn't hang together with the others or associate the same way with other variables). I analyzed the results using a continuous measure of LW exposure, but to simplify reporting, I'll give the results below by comparing those in the top third on this measure of LW exposure with those in the bottom third.
There were 5 intelligence-related measures which I combined into a single composite measure of Intelligence (SAT out of 2400, SAT out of 1600, ACT, previously-tested IQ, extra credit IQ test); I used this to control for intelligence and to compare the effects of LW exposure with the effects of Intelligence (for the latter, I did a similar split into thirds). Sample sizes: 1101 people answered at least one of the CFAR questions; 1099 of those answered at least one LW exposure question and 835 ...
(more detailed results, continued)
Question 4: Imagine that you are a doctor, and one of your patients suffers from migraine headaches that last about 3 hours and involve intense pain, nausea, dizziness, and hyper-sensitivity to bright lights and loud noises. The patient usually needs to lie quietly in a dark room until the headache passes. This patient has a migraine headache about 100 times each year. You are considering three medications that you could prescribe for this patient. The medications have similar side effects, but differ in effectiveness and cost. The patient has a low income and must pay the cost because her insurance plan does not cover any of these medications. Which medication would you be most likely to recommend?
- Drug A: reduces the number of headaches per year from 100 to 30. It costs $350 per year.
- Drug B: reduces the number of headaches per year from 100 to 50. It costs $100 per year.
- Drug C: reduces the number of headaches per year from 100 to 60. It costs $100 per year.
This question is based on research on the decoy effect (aka "asymmetric dominance" or the "attraction effect"). Drug C is obviously worse than Drug B (it is st...
ALTERNATIVE POLITICS QUESTION: Progressive: 429, 36.3% Libertarian: 278, 23.5% Reactionary: 30, 2.5% Conservative: 24, 2% Communist: 22, 1.9% Other: 156, 13.2%
I'd like to note that my suggestion as I offered it didn't include an "Other" option -- you added that one by yourself, and it ended up being selected by more people than "Reactionary" "Conservative" and "Communist" combined. My suggested question would have forced the current "Others" to choose between the five options provided or not answer at all.
Fishing for correlations is a statistically dubious practice, but also fun. Some interesting ones (none were very high, except e.g. Father Age and Mother Age):
- IQ and Hours Writing have correlation 0.26 (75 degrees), which is the only interesting IQ correlation.
- Siblings and Older siblings have correlation 0.48 (61 degrees), which isn't too surprising , but makes me wonder: do we expect this correlation to be 0.5 in general?
- Most of the Big Five answers are slightly correlated (around +/-0.25, or 90+/-15 degrees) with each other, but not with anything els
CFAR question 7 (guess of height of redwood) was negatively correlated with Height (-0.23, or 103 degrees). No notable correlation with the random number, though.
I looked at this with the data set that I used for my CFAR analyses, and this correlation did not show up; r=-.02 (p=.58). On closer inspection, the correlation is present in the complete un-cleaned-up data set (r=-.21), but it is driven entirely by a single outlier who listed their own height as 13 cm and the height of the tallest redwood as 10,000 ft.
(In my analyses of the anchoring question I had excluded the data of 2 outliers who listed redwood heights of 10,000 ft. and 1 ft. Before running this correlation with Height, I checked the distribution and excluded everyone who listed a height under 100 cm, since those probably represent units confusions.)
I was surprised to see that LW has almost as many socialists as libertarians. I had thought due to anecdotal evidence that the site was libertarian-dominated.
I was also surprised that a plurality of people preferred dust specks to torture, given that it appears to be just a classic problem of scope insensitivity, which this site talks about repeatedly.
I was happy to see that we have more vegetarians and fewer smokers than the general population.
Generally, half the time we get visiting leftwingers accusing us of being rightwing reactionaries, and the other half of the time we get visiting rightwingers accusing us of being leftwing sheep.
So if you thought that the site was libertarian-dominated, I'm hereby making a prediction with 75% certainty that you consider yourself a left winger. Am I right?
There are a number of old posts from the Overcoming Bias days in which EY comments that the audience is primarily libertarian- which makes sense for the blog of a GMU economist. A partial explanation might be people reading that and assuming he's talking about the modern population distribution of LW.
Oh yes, that reminds me - I've always wondered if MoR was a waste of time or not in terms of community-building. So let's divide the dataset into people who were referred to LW by MoR and people who weren't...
Summary: they are younger, lower karma, lower karma per month participating (karma log-transformed or not), more likely to be students; but they have the same IQ (self-report & test) as the rest.
So, Eliezer is successfully corrupting the youth, but it's not clear they are contributing very much yet.
R> lw <- read.csv("lw-survey/2012.csv")
R> hpmor <- lw[as.character(lw$Referrals) == "Referred by Harry Potter and the Methods of Rationality",]
R> hpmor <- lw[as.character(lw$Referrals) != "Referred by Harry Potter and the Methods of Rationality",]
R> t.test(hpmor$IQ, hpmor$IQ)
Welch Two Sample t-test
data: hpmor$IQ and hpmor$IQ
t = 0.5444, df = 99.28, p-value = 0.5874
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-2.614 4.591
sample estimates:
mean of x mean of y
139.1 138.1
R> t.test(as.integer(as.character(hpmor$IQTest)), as.integer(as.character(hpmor$- We are less likley to hear the strongest arguments in favour of those political positions
- We are less likely to realize we are straw manning a position
- Convenient but unjustified assumptions are less likely to be called out
- Our thought and speculation about ethics and values in humans will be skewed
- Conservative rationalists feel excluded from the intended audience
- Since there are many many more people in the world who hold "conservative" positions than "progressive" ones we may not be properly mining a source of valuable community members.
In other words the standard pro-diversity arguments apply and arguably they applies more strongly than for some other categories it has been invoked for. I think value and political diversity is one of the best ways to as a community be able to detect motivated cognition.
I will flat out say that I think people with reactionary view points from the past 200 years have had a remarkable prescience in predicting outcomes. It is simply that once those outcomes come about we don't consider them bad any more, indeed we develop sacred feelings around them.
Assume you agree with all changes that occurred in the mentioned time period. Indeed assume you agree with the changes that are likely to occur in the next 20 years as well. Unless you have a good reason to believe "moral progress" is coherent and happening right now history has shown there is literally no way from preventing inhuman processes of memetic and biological evolution from grinding down your complex values.
This should be deeply disturbing.
The first one surprises me because hardly anyone on LW seems conservative (and the polls confirm this).
Just in the last two or three months I remember there was one guy that accused us of being the right-wing conspiracy of multibillionaire Peter Thiel (because he has donated money to the Singularity Instittue), a few who accused the whole site of transphobia (for not banning a regular user for a bigoted joke he made on a different location, not actually in this forum), one who called us the equivalent of fascist sheep (for having more people here read Mencius Moldbug on average than I suppose is the internet forum median)...
Fanatics view tolerance of the enemy as enemy action. So, yeah, I think leftwing fanatics will view anything even tolerant of either reaction or libertarianism as their enemy -- even as they don't notice that similar tolerance is extended to positions to the left of them.
The first one surprises me because hardly anyone on LW seems conservative (and the polls confirm this).
However, there are a few fairly common (or at least it seems so to me) opinions on LW which are distinctively un-Left: democracy is bad, there are racial differences in important traits, and women complain way too much about how men treat them. We'll see how that last one plays out.
there are a few fairly common (or at least it seems so to me)
I think that they appear to be more common than they actually are because their proponents are much louder than everyone else.
democracy is bad, there are racial differences in important traits, and women complain way too much about how men treat them
One of those is a factual question, not a policy question. (Also, there are plenty of left-wingers who wouldn't throw a fit at “it appears that black people and Native Americans have lower average IQ than white people, whereas East Asians and Ashkenazi Jews have higher average IQ; the differences between the group averages are comparable with the standard deviations within each group; it's not yet fully clear how much the differences between the group averages are due to genetics and how much they are due to socio-economic conditions”, at least outside liberal arts departments.)
You don't think that the "birther controversy" was racist in nature?
No.
You think this whole thing is a coincidence?
Racists do not generally like Obama because he has an African father. This is incredibly surprising? Do a google search for racist epithets and say Condoleezza Rice.
Are you really saying that if Democrats had a white candidate on election day 2016 and Republicans a black one, you wouldn't find the appropriate slurs online?
You think this type of thing doesn't happen?
Of course it does. Do you have evidence it happens often enough to be a concern rather than pointing out anecdotes? Also what has this got to do with Republicans rather than American society as a whole?
You think this is a complete fabrication?
No why would I? A mere two decades before that there was basically a new splinter party just on the segregation issue.
This seems like a complete failure of critical thinking.
I will say the same. You seem to have utterly failed at the principle of charity if not outright straw manned my position.
I am not an American. I'm not a Republican. Yet the examples you picked seem to target a CNN caricature of a Fox News viewer. Why did you pat...
Thank you for the correction! I've put the words in my Anki deck. English is not my native language so I often make mistakes, please if you ever spot an error don't hesitate to comment or PM me.
Also, I find the birther controversy weird because I think some laws matter a great deal more than others, and the natural born citizen rule doesn't serve any important purpose that I can see.
I asked a birther what he expected to happen if it turned out that Obama was proven to have been born in Kenya, and he hadn't even thought about the question, which probably implied that few if any birthers had thought about it either, or they would have been discussing it. I'm not sure this proves anything about racism, but it's evidence that there was something weird going on.
What disturbs you about this specifically?
Like I already said a few times, nearly all the highly upvoted posts and comments that explicitly bring up ideology - like yours - appear to come from the right. Duh, you'll say, if most of the LW stuff is implicitly liberal/progressive, then of course what's going to stand out is (intelligently argued) contrarianism. But the disturbing thing to me is that the mainstream doesn't seem to react to the challenge.
What I have in mind is not some isolated insightful comments e.g. criticizing moldbuggery, defending egalitarianism or feminism or something like that - they do appear - but an acknowledgement of LW's underlying ideological non-neutrality. E.g. this post by Eliezer, or this one by Luke would've hardly been received well without the author and the audience sharing Enlightenment/Universalist values; both the tone and the message rely on an ideological foundation (one that I desire to analyze and add to - not deconstruct).
Yet there's not enough acknowledgement and conscious defense of those values, so when such content is challenged from an alt-right perspective, the attacking side ends up with the last word in the discussion. So to ...
The thing is right wing thinkers who end up on LessWrong and stay in the community should be comforting to you, these are the people who believe engaging in dialogue and common goals is possible. And I would argue they empower all members of the community by contributing to the explicit goal of refining human rationality or FAI design (though they might undermine some other implicit goals).
Compare this to the idea of right wing thinkers that take what they can from rationality and the alt right and then seeing they are not accepted in the nominally rationalist community leave for the world. Even as individuals that should concern you, but imagine a right wing community forming powered by the best tools from here. Somehow it seems its left wing only counterpart would be weaker.
their values
You are too quick in ascribing incompatible values to people you disagree with. That's the cheap way out; it allows you to write off their opinion without considering the fact that they might have the same terminal values as you, and arrived at their instrumental position for rational, empirical reasons. Then you'd have to actually consider whether their position is correct, instead of just writing them off.
Extreme right-wingers all seem to share the explicit values of institutionalized dominance, rigid hierarchy, rejection of universal ethics and the suppression of any threat to such an order.
This is the straw-man version you get taught about by the Universalist establishment. Don't take it seriously as what these folks are actually thinking. Some people are just dumb and evil, and most confuse "this is instrumentally a good idea" with "this is terminally a good idea" but there's less of them than you are taught, and there actually are good reasons for the apparent craziness.
It is perfectly possible for someone to have the same values as you and consider (the non-straw) version of those things to be instrumentally a good idea.
...I have pondere
As someone who finds alt-right ideas interesting to read about and discuss, but is at the end of the day a conventional mainstream liberal, the advice I'd give you is: you should chill out.
Discussion of political topics at this site, as at Moldbug's and other related ones, and also the vast majority of blogs and sites all over the political spectrum (with the possible but tiny exception of a handful of blogs connected to the D or R party apparatus or to insiders affecting government policy decisions) is essentially mental masturbation, something that will not affect in any way the future of humanity. It is just a way to pass the time some find interesting, as others prefer solving Sudoku puzzles or pondering Newcomblike problems.
Your feeling that a group of ideological "outsiders" who don't share your values is growing in influence, and might take over if they are not "cordoned" and lead to some horrible catastrophe, sounds like the kind of feeling appropriate for a small hunter-gatherer tribe where if a dozen or two enemies of you join forces and take over you will have a very bad time. It is not appropriate for the objective situation of a forum with several thousand people, and much less for a country of 300 million people or a humanity of 7 billion people. The future of the world, even the future of LW, is not going to be shaped by the occasional crypto-racist (/sexist/fascist/etc) posts of a handful of people.
The only proper object of blame is Mother Nature; and she is capable of inflicting far worse things on us than mere statistical disparities between ancient inbred populations..
This does not seem like submission to nature to me. I do not think he would object at all to say genetic engineering or eugenic programs aimed at reducing such suffering or boosting cognitive performance.
This might fit my definition of "reason"... but what is noble or compassionate about it? How is Derbyshire preaching acceptance of inequality and submission to Nature different from the Catholic Church preaching acceptance of death and submission to God? If you think it reasonable to loathe death, why would you not loathe the genetic lottery?
Derbyshire is asking us to please stop trying things that do not work and scapegoating those who aren't responsible for misery inflicted by nature! I find it remarkable that you do not seem to grasp the moral relevance of avoiding scapegoating people at all! It is a terrible thing to look down on people and make them feel guilty and bad for something that is not their fault.
If you want to find nobility and compassion I say look here.
...Under a reigning ph
They are. They just can't come up with good arguments.
I think what is happening here is a bit more subtle than your summary suggests. First, many of the notions being proposed or discussed while in some sense "conservative'' are things like Moldbug's ideas which while they do fall into one end, they aren't in any way standard arguments or even issues. So people may simply not be able to raise effective arguments since they are grappling with approaches with which they haven't had to think about before. Similarly, I suspect that all of us would have trouble making responses to arguments favoring say complete dissolution of all governments above the county level, not because such arguments are strong, but because we're not used to thinking about them or constructing arguments against them.
Moreover, the meta-contrarian nature of Less Wrong, makes people very taken with arguments of forms that they haven't seen before, so there may be a tendency to upvote or support an interesting contrarian argument even as one doesn't pay as much attention to why the argument simply fails.
Finally, contrarian attitudes have an additional advantage when phrased in a political context: They aren't as obviously political. The politics-as-mindkiller meme is very strong here, so a viewpoint that everyone recognizes as by nature political gets labeled as potential mindkilling to be avoided while arguments that don't fit into the standard political dialogue as much don't pattern match as closely.
We have seen posters motivated enough to engage in karmassasination of users making right wing arguments so this seems plausible.
It seems possibly relevant to point out that karma assasination has been occurring in the last few days to people of a wide variety of political viewpoints. For example the recent thread on women's experience was reported as leading to multiple incidents of karma assassination to people espousing views classically labeled as feminist.
Engagement in karma fights probably doesn't give much data about accuracy of beliefs or peoples confidence in their own belief structures.
I was also surprised that a plurality of people preferred dust specks to torture, given that it appears to be just a classic problem of scope insensitivity, which this site talks about repeatedly.
I was surprised as well, but I disagree that it is necessarily scope insensitivity - believing utility is continuously additive requires choosing torture. But some people take that as evidence that utility is not additive - more technically, evidence that utility is not the appropriate analysis of morality (aka picking deontology or virtue ethics or somesuch).
These are all the countries with greater than 1% of Less Wrongers,
And they are exactly the non-write-in ones in the survey, except for New Zealand that was there and Poland that wasn't.
I didn't do this myself because I didn't trust my statistical ability enough, and I forgot to mention it on the original post, but...
Can someone check for birth order effects? Whether Less Wrongers are more likely to be first-borns than average? Preferably someone who's read Judith Rich Harris' critique of why most birth order effect analyses are hopelessly wrong? Or Gwern? I would trust Gwern on this.
I don't know Harris's critique, but here are some numbers.
Out of survey respondents who reported that they have 1 sibling (n=453), 76% said that they were the oldest (i.e., 0 older siblings). By chance, you'd expect 50% to be oldest.
Of those with 2 siblings, 50% are the oldest (vs. 33% expected by chance), n=240.
Of those with 3 siblings, 45% are the oldest (vs. 25% expected by chance), n=120.
Of those with 4 or more siblings, 50% are the oldest (vs. under 20% expected by chance), n=58.
Of those with 0 siblings, 100% are the oldest (vs. 100% expected by chance), n=163.
Overall, 69% of those who answered the "number of older siblings" question are the oldest.
Those look like big effects, unlikely to be explained by whatever artifacts Harris has found.
There are a handful of people who left the number of older siblings blank but did report a total number of siblings), or who reported a non-integer number of siblings (half-siblings), but they are too few to make much difference in the numbers.
This doesn't seem to vary by degree of involvement in LW; overall 71% of those in the top third of LW exposure (based on sequence-reading, karma, etc.) are the oldest. Here is a little tabl...
There seems to be a pretty big potential confounder: age. Many respondents' younger siblings are too young to be contributing to this site, while no one's older siblings are too old (unless they're dead, but since ~98% of the community is under age 60 that's not a significant concern).
You're saying that if we randomly picked 22-31 year-olds, a disproportionate member would be eldest children? For that to work, there'd have to be more eldest children in that age-range than youngest. Given the increase in population, that is certainly plausible. You would expect more younger families than older families, which means that within an age range there would be a disproportionate number of older siblings (unless it's so young that not all of the younger siblings have been born yet) but it doesn't seem like it would be nearly that significant.
Many respondents' younger siblings are too young to be contributing to this site, while no one's older siblings are too old
The fact that most of the respondents are eldest children is a confounder for this.
(unless they're dead, but since ~98% of the community is under age 60 that's not a significant concern).
In that case, wouldn't people over 60 also be too old?
I had no knowledge of such a survey. These might be more efficient if they were posted in a blatantly obvious manner, like on the banner.
IQ Trend Analysis:
The self-reported IQ results on these surveys have been, to use Yvain's wording, "ridiculed" because they'd mean that the average LessWronger is gifted. Various other questions were added to the survey this time which gives us things to check against, and the results of these other questions have made the IQ figures more believable.
Summary:
LessWrong has lost IQ points on the self-reported scores every year for a total of 7.18 IQ points in 3.7 years or about 2 points per year. If LessWrong began with 145.88 IQ points in May 20...
As I mentioned previously, and judging from the graphs, the standard deviations of the IQs are obviously mixed up, because they were not determined in the questionnaire, and probably people who answered are not educated about them either. Including IQs in s.d. 24 with those in s.d. 16 and 15 is bound to inflate the average IQ. The top scores in that graph, or at the very least some of them, are in s.d. 24, which means that they would be a lot lower in s.d. 15. IQ 132 is the cutoff for s.d. 16, while s.d. 15 is the one most adopted in recent scientific literature. For s.d. 24, it is 148. Mensa and often people on the press like to use s.d. 24 to sound more impressive to amateurs.
This probably makes tests like the SAT more reliable as an estimation, because they have the same standard for all who submitted their scores, although in this case the ceiling effect would become apparent, because perfect or nearly-perfect scores wouldn't go upwards of a certain IQ.
As for the IQ question and especially the self-reported IQ, it did not take into account that IQ should come at least with standard deviation. Otherwise it's like asking for a height number without saying if it is in centimeters, meters, or feet. It's understandable that people who didn't study psychometrics with some depth don't know this, though.
IQ can be a ratio IQ or a deviation IQ. In the first case it is mental age divided by actual age, with the normalcy as 100. This is still used mostly for children, but it's still possible to see such scores. Devi...
I think you missed some duplicates in for_public.csv: Rows 26, 30, 761 and 847 are identical to their preceding one.
Note that people who take online psychometric tests are probably a pretty skewed category already so this tells us nothing.
What? They calibrated the test using the people who took it online?
Not a survey response but too good to omit:
The website you mentioned, namely lesswrong.com is typical of the rationalists. If you read the articles posted on the site, you can see how they push their rationalist methods even beyond the limits of reason. From the Islamic point of view, one may say that their overdependence on rationality amounts to a sort of self-sufficiency and arrogance which smacks of nothing less than the Satanic, as Satan or Iblis is the epitome of arrogance.
http://www.onislam.net/english/ask-about-islam/islam-and-the-world/worldvi...
I wouldn't necessarily read too much into your calibration question, given that it's just one question, and there was something of a gotcha.
One thing I learned from doing calibration exercises is that I tended to be much too tentative with my 50% guesses.
When I answered the calibration question, I used my knowledge of other math that either had to, or couldn't have come before him, to narrow the possible window of his birth down to about 200 years. Random chance would then give me about a 20% shot. I thought I had somewhat better information than random...
Now I wish I had written a funnier description...so many of these are silly~
Alternate Explanations for LW's Calibration Atrociousness:
Maybe a lot of the untrained people simply looked up the answer to the question. If you did not rule that out with your study methods, then consider seeing whether a suspiciously large number of them entered the exact right year?
Maybe LWers were suffering from something slightly different from the overconfidence bias you're hoping to detect: difficulty admitting that they have no idea when Thomas Bayes was born because they feel they should really know that.
The mean was 1768, the median 1780, and the mode 1800. Only 169 of 1006 people who answered the question got an answer within 20 years of 1701. Moreover, the three people that admitted to looking it up (and therefore didn't give a calibration) all gave incorrect answers: 1750, 1759, and 1850. So it seems like your first explanation can't be right.
After trying a bunch of modifications to the data, it seems like the best explanation is that the poor calibration happened because people didn't think about the error margin carefully enough. If we change the error margin to 80 years instead of 20, then the responses seem to look roughly like the untrained example from the graph in Yvain's analysis.
Another observation is that after we drop the 45 people who gave confidence levels >85% (and in fact, 89% of them were right), the remaining data is absolutely abysmal: the remaining answers are essentially uncorrelated with the confidence levels.
This suggests that there were a few pretty knowledgeable people who got the answer right and that was that. Everyone else just guessed and didn't know how to calibrate; this may correspond to your second explanation.
If we really want to test them, we need to make it clear that we're testing them. And if we want them to be serious about it, we have to make it clear that it's important.
Uh, what? The point of LessWrong is to make people better all the time, not just better when they think "ah, now it's time to turn on my rationality skills." If people aren't applying those skills when they don't know they're being tested, that's a very serious problem, because it means the skills aren't actually ingrained on the deep and fundamental level that we want.
You know that, Katydee, but do all the people who are taking the survey think that way? The majority of them haven't even finished the sequences. I agree with you that it's ideal for us to be good rationalists all the time, but mental stamina is a big factor.
Being rational takes more energy than being irrational. You have to put thought into it. Some people have a lot of mental energy. To refer to something less vague and more scientific: there are different levels of intelligence and different levels of intellectual supersensitivity (A term from Dabrowski that refers to how excitable certain aspects of your nervous system are.) Long story short: Some people cannot analyze constantly because it's too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit. If that's the case, it probably seems strange to you that anybody wouldn't attempt to maintain a state of constant analysis. Most people with unusual intellectual stamina seem to view others as lazy when they observe that those other people aren't doing intellectual things all the time. It frequently does ...
Long story short: Some people cannot analyze constantly because it's too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit.
Fascinating!
It's making me realize why my summer project, which was to read Eat That Frog by Brian Tracey, was such a failure. The book is meant to be applied to work, preferably in an office environment–i.e. during your 40 productive work-hours. I was already working 40 hours a week at my extremely stimulating job as a nurse's aid at the hospital, where I had barely any time to sit down and think about anything, and I certainly didn't have procrastination problems. Then I would get home, exhausted with my brain about to explode from all the new interesting stuff I'd been seeing and doing all day, and try to apply Brian Tracey's productivity methods to the personal interest projects I was doing in my spare time.
This was a very efficient way to make these things not fun, make me feel guilty about being a procrastinator, etc. It gave me an aversion to starting projects, because the part of my brain that likes and needs to do something easy ...
you're actually taking the time to distinguish between 10 different amounts of confidence (10%, 20%, 30%, etc), and then making ten more tiny distinctions (30%, 31%, 32% for instance)... at least that's the way that I do it
The straightforward interpretation of your words evaluates as a falsity, as you can't estimate informal beliefs to within 1%.
It could be that many people self-reported IQ based off of their SAT or ACT scores, which would explain away the correlation. How many people reported both SAT and ACT scores?
About 25% of cis women who answered the question are vegetarian, compared to about 12.5% of cis men. This is much less extreme than among people I've met in person (only 2 men that I remember of, vs at least 10 women).
How many of us are there:
A couple of months ago, I asked Trike, the company that manages the website, for a complete list of LessWrong registration dates in order to make a growth chart. I received it on 08-23-2012. The data shows that LessWrong has 13,727 total users, not including spammers and accounts that were deleted.
See also: LessWrong Growth Bar Graph (in the thread "Preventing discussion from being watered down by an "endless September" user influx.")
Problem:
The line: "This includes all types with greater than 10 people. You can see the full table here." links to a gif that is inaccurate, has no key to explain oddities, and is of such poor graphical quality that parts of it are actually unreadable.
It may be that the reason that invalid personality types like "INNJ" are listed is due to typos on the part of the survey participants. If so, then great! But it may also be that the person who constructed this graphic put typos in (I consider this fairly likely due to the fact that the...
Women were on average newer to the community - 21 months vs. 39 for men - but to my surprise a t-test was unable to declare this significant. Maybe I'm doing it wrong?
Well, possibly. The t-distribution is used for "estimating the mean of a normally distributed population," (yay wikipedia) and you're trying to estimate the mean of a slanted-uniformly-distributed-with-a-spike-at-the-beginning population.
But there is another important consideration, which is that applying more scrutiny to unexpected results gives you systematic error (confirmati...
Well-educated atheist American white men in their mid 20s with no children who work with computers.
"The new thing for people who would have been Randian Objectivists 30 years ago."
The demographics are essentially the same except LW is probably more than 2:1 politically left vs. right. Objectivists are probably more than 2:1 in the other direction.
Since when did people like us decide it is OK to be liberal/socialist?
By the way, has anyone figured out how to load the CSV in R? read.table chokes with problems like "Error in read.table("for_public.csv", header = TRUE, quote = "", sep = ",") : more columns than column names"; dos2unix doesn't help, and loading it up in OpenOffice, it looks fine but re-exporting as CSV leads to the same darn errors. Mucking about, I can get it to load if I delete everything but the first entry and then the last 4 commas but this solution doesn't work for any additional entries!
use "as.numeric(as.character(dat$IQTest))"
The IQtest data is stored as factor. A factor variable has a set of levels, numbered 1,2,3,... that are the variable can possibly take on and labels for those factors. as.numeric(X) returns the level numbers of X. as.character returns the labels of X. In the case that the labels are actually numbers (usually integers that R is interpreting as character labels for some reason), as.numeric(as.character(X)) will return the numeric values that R is interpreting as labels.
EDIT:
In this case, when no value for IQtest was reported, it was stored as " " instead of "", which made R think the variable contained character data which R defaults to treating as factors. The " "'s should all be NA's once it's converted properly.
(Nearly deleting this post due to my idiocy. I can't read column titles correctly.)
SAT1600 takers: 8% theist
SAT2400 takers: 0% theist
Income-wise, there seem to be some anomalies and I'm not sure what factors we ought to expect to be driving them or what is privileging a hypothesis. So here's a poll asking your opinion on which of the correlates we should expect to be most important in governing LWers' income (given in alphabetical order):
[pollid:260]
Sweet. I was in the one correctly calibrated cohort - I knew just how slim my chances of being right were!
On the reason for the absence of multiple comparison correction in my various quick tests here: http://lesswrong.com/lw/h56/the_universal_medical_journal_article_error/8q28
A question arose on #lesswrong as to whether female LWers might be more likely to find LW through MoR than not. There is an imbalance in MoR referrals by gender, but it's not sufficiently extreme to hit significance in the limited survey dataset (need moar women):
R> lw <- read.csv("2012.csv")
R> lwm <- subset(lw, as.character(lw$Gender) == "M (cisgender)")
R> lwf <- subset(lw, as.character(lw$Gender) == "F (cisgender)")
R> summary(lwm$HPMOR)
" " Another analysis: t-test/logistic regression does not indicate a relationship between getting the first CFAR logic puzzle right and having answered more survey questions than those who got the logic question wrong. (Tim Tyler suggested that there might be a commitment/rushing effect.)
10 people said "Drug C: reduces the number of headaches per year from 100 to 60. It costs $100 per year" over "Drug B: reduces the number of headaches per year from 100 to 50. It costs $100 per year" on CFAR question #4...
I said "Drug A: reduces the number of headaches per year from 100 to 30. It costs $350 per year" personally. I think there's a case for B, maybe, but who picks C?
Are people who understand quantum mechanics are more likely to believe in Many Worlds? We perform a t-test, checking whether one's probability of the MWI being true depends on whether or not one can solve the Schrodinger Equation. People who could solve the equation had on average a 54.3% probability of MWI, compared to 51.3% in those who could not. The p-value is 0.26; there is a 26% probability this occurs by chance. Therefore, we fail to establish that people's probability of MWI varies with understanding of quantum mechanics.
Just wanted to point out...
If the correlation had come out the other way, you'd be jumping on it as proof of your thesis that LWers favor MWI because they are sheepishly following Eliezer. In what universe where they are indeed sheepishly and ignorantly following him does a question like that show nothing whatsoever?
I'm assuming that the question was meant as a simple and polite proxy for "Does your knowledge of quantum mechanics include some actual mathematical content, or is it just taken from popular science books and articles?"
The CSV seems to have different data than you reported. For example, it shows 63% theists and 18% atheists. It doesn't appear to be a simple reversal of labels.
God: 6 + 18.7 (0, 0, 1) [n = 1098] Simulation: 25.1 + 29.7 (1, 10, 50) [n = 1039]
From those who believe that we are in a simulation with over 70% confidence there's only one person who has a higher chance of God existing then the chance that we live in a simulation. Given that a God got here defined as someone with world making powers, how do you get a simulation without a God?
Look again at the survey questions:
P(Supernatural)
What is the probability that supernatural events, defined as those involving ontologically basic mental entities, have occurred since the beginning of the universe?P(God)
What is the probability that there is a god, defined as a supernatural (see above) intelligent entity who created the universe?
A simulator is not a god because gods are ontologically basic, while simulators are not.
Thank you to everyone who took the 2012 Less Wrong Survey (the survey is now closed. Do not try to take it.) Below the cut, this post contains the basic survey results, a few more complicated analyses, and the data available for download so you can explore it further on your own. You may want to compare these to the results of the 2011 Less Wrong Survey.
Part 1: Population
How many of us are there?
The short answer is that I don't know.
The 2011 survey ran 33 days and collected 1090 responses. This year's survey ran 23 days and collected 1195 responses. The average number of new responses during the last week was about five per day, so even if I had kept this survey open as long as the last one I probably wouldn't have gotten more than about 1250 responses. That means at most a 15% year on year growth rate, which is pretty abysmal compared to the 650% growth rate in two years we saw last time.
About half of these responses were from lurkers; over half of the non-lurker remainder had commented but never posted to Main or Discussion. That means there were only about 600 non-lurkers.
But I am skeptical of these numbers. I hang out with some people who are very closely associated with the greater Less Wrong community, and a lot of them didn't know about the survey until I mentioned it to them in person. I know some people who could plausibly be described as focusing their lives around the community who just never took the survey for one reason or another. One lesson of this survey may be that the community is no longer limited to people who check Less Wrong very often, if at all. One friend didn't see the survey because she hangs out on the #lesswrong channel more than the main site. Another mostly just goes to meetups. So I think this represents only a small sample of people who could justly be considered Less Wrongers.
The question of "how quickly is LW growing" is also complicated by the high turnover. Over half the people who took this survey said they hadn't participated in the survey last year. I tried to break this down by combining a few sources of information, and I think our 1200 respondents include 500 people who took last year's survey, 400 people who were around last year but didn't take the survey for some reason, and 300 new people.
As expected, there's lower turnover among regulars than among lurkers. Of people who have posted in Main, about 75% took the survey last year; of people who only lurked, about 75% hadn't.
This view of a very high-turnover community and lots of people not taking the survey is consistent with Vladimir Nesov's data showing http://lesswrong.com/lw/e4j/number_of_members_on_lesswrong/77xz 1390 people who have written at least ten comments. But the survey includes only about 600 people who have at least commented; 800ish of Vladimir's accounts are either gone or didn't take the census.
Part 2: Categorical Data
SEX:
Man: 1057, 89.2%
Woman: 120, 10.1%
Other: 2, 0.2%)
No answer: 6, 0.5%
GENDER:
M (cis): 1021, 86.2%
F (cis): 105, 8.9%
M (trans f->m): 3, 0.3%
F (trans m->f): 16, 1.3%
Other: 29, 2.4%
No answer: 11, 0.9%
ORIENTATION:
Heterosexual: 964, 80.7%
Bisexual: 135, 11.4%
Homosexual: 28, 2.4%
Asexual: 24, 2%
Other: 28, 2.4%
No answer: 14, 1.2%
RELATIONSHIP STYLE:
Prefer monogamous: 639, 53.9%
Prefer polyamorous: 155, 13.1%
Uncertain/no preference: 358, 30.2%
Other: 21, 1.8%
No answer: 12, 1%
NUMBER OF CURRENT PARTNERS:
0: 591, 49.8%
1: 519, 43.8%
2: 34, 2.9%
3: 12, 1%
4: 5, 0.4%
6: 1, 0.1%
7, 1, 0.1% (and this person added "really, not trolling")
Confusing or no answer: 20, 1.8%
RELATIONSHIP STATUS:
Single: 628, 53%
Relationship: 323, 27.3%
Married: 220, 18.6%
No answer: 14, 1.2%
RELATIONSHIP GOALS:
Not looking for more partners: 707, 59.7%
Looking for more partners: 458, 38.6%
No answer: 20, 1.7%
COUNTRY:
USA: 651, 54.9%
UK: 103, 8.7%
Canada: 74, 6.2%
Australia: 59, 5%
Germany: 54, 4.6%
Israel: 15, 1.3%
Finland: 15, 1.3%
Russia: 13, 1.1%
Poland: 12, 1%
These are all the countries with greater than 1% of Less Wrongers, but other, more exotic locales included Kenya, Pakistan, and Iceland, with one user each. You can see the full table here.
This data also allows us to calculate Less Wrongers per capita:
Finland: 1/366,666
Australia: 1/389,830
Canada: 1/472,972
USA: 1/483,870
Israel: 1/533,333
UK: 1/603,883
Germany: 1/1,518,518
Poland: 1/3,166,666
Russia: 1/11,538,462
RACE:
White, non-Hispanic 1003, 84.6%
East Asian: 50, 4.2%
Hispanic 47, 4.0%
Indian Subcontinental 28, 2.4%
Black 8, 0.7%
Middle Eastern 4, 0.3%
Other: 33, 2.8%
No answer: 12, 1%
WORK STATUS:
Student: 476, 40.7%
For-profit work: 364, 30.7%
Self-employed: 95, 8%
Unemployed: 81, 6.8%
Academics (teaching): 54, 4.6%
Government: 46, 3.9%
Non-profit: 44, 3.7%
Independently wealthy: 12, 1%
No answer: 13, 1.1%
PROFESSION:
Computers (practical): 344, 29%
Math: 109, 9.2%
Engineering: 98, 8.3%
Computers (academic): 72, 6.1%
Physics: 66, 5.6%
Finance/Econ: 65, 5.5%
Computers (AI): 39, 3.3%
Philosophy: 36, 3%
Psychology: 25, 2.1%
Business: 23, 1.9%
Art: 22, 1.9%
Law: 21, 1.8%
Neuroscience: 19, 1.6%
Medicine: 15, 1.3%
Other social science: 24, 2%
Other hard science: 20, 1.7%
Other: 123, 10.4%
No answer: 27, 2.3%
DEGREE:
Bachelor's: 438, 37%
High school: 333, 28.1%
Master's: 192, 16.2%
Ph.D: 71, 6%
2-year: 43, 3.6%
MD/JD/professional: 24, 2%
None: 55, 4.6%
Other: 15, 1.3%
No answer: 14, 1.2%
POLITICS:
Liberal: 427, 36%
Libertarian: 359, 30.3%
Socialist: 326, 27.5%
Conservative: 35, 3%
Communist: 8, 0.7%
No answer: 30, 2.5%
You can see the exact definitions given for each of these terms on the survey.
RELIGIOUS VIEWS:
Atheist, not spiritual: 880, 74.3%
Atheist, spiritual: 107, 9.0%
Agnostic: 94, 7.9%
Committed theist: 37, 3.1%
Lukewarm theist: 27, 2.3%
Deist/Pantheist/etc: 23, 1.9%
No answer: 17, 1.4%
FAMILY RELIGIOUS VIEWS:
Lukewarm theist: 392, 33.1%
Committed theist: 307, 25.9%
Atheist, not spiritual: 161, 13.6
Agnostic: 149, 12.6%
Atheist, spiritual: 46, 3.9%
Deist/Pantheist/Etc: 32, 2.7%
Other: 84, 7.1%
RELIGIOUS BACKGROUND:
Other Christian: 517, 43.6%
Catholic: 295, 24.9%
Jewish: 100, 8.4%
Hindu: 21, 1.8%
Traditional Chinese: 17, 1.4%
Mormon: 15, 1.3%
Muslim: 12, 1%
Raw data is available here.
MORAL VIEWS:
Consequentialism: 735, 62%
Virtue Ethics: 166, 14%
Deontology: 50, 4.2%
Other: 214, 18.1%
No answer: 20, 1.7%
NUMBER OF CHILDREN
0: 1044, 88.1%
1: 51, 4.3%
2: 48, 4.1%
3: 19, 1.6%
4: 3, 0.3%
5: 2, 0.2%
6: 1, 0.1%
No answer: 17, 1.4%
WANT MORE CHILDREN?
No: 438, 37%
Maybe: 363, 30.7%
Yes: 366, 30.9%
No answer: 16, 1.4%
LESS WRONG USE:
Lurkers (no account): 407, 34.4%
Lurkers (with account): 138, 11.7%
Posters (comments only): 356, 30.1%
Posters (comments + Discussion only): 164, 13.9%
Posters (including Main): 102, 8.6%
SEQUENCES:
Never knew they existed until this moment: 99, 8.4%
Knew they existed; never looked at them: 23, 1.9%
Read < 25%: 227, 19.2%
Read ~ 25%: 145, 12.3%
Read ~ 50%: 164, 13.9%
Read ~ 75%: 203, 17.2%
Read ~ all: 306, 24.9%
No answer: 16, 1.4%
Dear 8.4% of people: there is this collection of old blog posts called the Sequences. It is by Eliezer, the same guy who wrote Harry Potter and the Methods of Rationality. It is really good! If you read it, you will understand what we're talking about much better!
REFERRALS:
Been here since Overcoming Bias: 265, 22.4%
Referred by a link on another blog: 23.5%
Referred by a friend: 147, 12.4%
Referred by HPMOR: 262, 22.1%
No answer: 35, 3%
BLOG REFERRALS:
Common Sense Atheism: 20 people
Hacker News: 20 people
Reddit: 15 people
Unequally Yoked: 7 people
TV Tropes: 7 people
Marginal Revolution: 6 people
gwern.net: 5 people
RationalWiki: 4 people
Shtetl-Optimized: 4 people
XKCD fora: 3 people
Accelerating Future: 3 people
These are all the sites that referred at least three people in a way that was obvious to disentangle from the raw data. You can see a more complete list, including the long tail, here.
MEETUPS:
Never been to one: 834, 70.5%
Have been to one: 320, 27%
No answer: 29, 2.5%
CATASTROPHE:
Pandemic (bioengineered): 272, 23%
Environmental collapse: 171, 14.5%
Unfriendly AI: 160, 13.5%
Nuclear war: 155, 13.1%
Economic/Political collapse: 137, 11.6%
Pandemic (natural): 99, 8.4%
Nanotech: 49, 4.1%
Asteroid: 43, 3.6%
The wording of this question was "which disaster do you think is most likely to wipe out greater than 90% of humanity before the year 2100?"
CRYONICS STATUS:
No, don't want to: 275, 23.2%
No, still thinking: 472, 39.9%
No, procrastinating: 178, 15%
No, unavailable: 120, 10.1%
Yes, signed up: 44, 3.7%
Never thought about it: 46, 3.9%
No answer: 48, 4.1%
VEGETARIAN:
No: 906, 76.6%
Yes: 147, 12.4%
No answer: 130, 11%
For comparison, 3.2% of US adults are vegetarian.
SPACED REPETITION SYSTEMS
Don't use them: 511, 43.2%
Do use them: 235, 19.9%
Never heard of them: 302, 25.5%
Dear 25.5% of people: spaced repetition systems are nifty, mostly free computer programs that allow you to study and memorize facts more efficiently. See for example http://ankisrs.net/
HPMOR:
Never read it: 219, 18.5%
Started, haven't finished: 190, 16.1%
Read all of it so far: 659, 55.7%
Dear 18.5% of people: Harry Potter and the Methods of Rationality is a Harry Potter fanfic about rational thinking written by Eliezer Yudkowsky (the guy who started this site). It's really good. You can find it at http://www.hpmor.com/.
ALTERNATIVE POLITICS QUESTION:
Progressive: 429, 36.3%
Libertarian: 278, 23.5%
Reactionary: 30, 2.5%
Conservative: 24, 2%
Communist: 22, 1.9%
Other: 156, 13.2%
ALTERNATIVE ALTERNATIVE POLITICS QUESTION:
Left-Libertarian: 102, 8.6%
Progressive: 98, 8.3%
Libertarian: 91, 7.7%
Pragmatist: 85, 7.2%
Social Democrat: 80, 6.8%
Socialist: 66, 5.6%
Anarchist: 50, 4.1%
Futarchist: 29, 2.5%
Moderate: 18, 1.5%
Moldbuggian: 19, 1.6%
Objectivist: 11, 0.9%
These are the only ones that had more than ten people. Other responses notable for their unusualness were Monarchist (5 people), fascist (3 people, plus one who was up for fascism but only if he could be the leader), conservative (9 people), and a bunch of people telling me politics was stupid and I should feel bad for asking the question. You can see the full table here.
CAFFEINE:
Never: 162, 13.7%
Rarely: 237, 20%
At least 1x/week: 207, 17.5
Daily: 448, 37.9
No answer: 129, 10.9%
SMOKING:
Never: 896, 75.7%
Used to: 1-5, 8.9%
Still do: 51, 4.3%
No answer: 131, 11.1%
For comparison, about 28.4% of the US adult population smokes
NICOTINE (OTHER THAN SMOKING):
Never used: 916, 77.4%
Rarely use: 82, 6.9%
>1x/month: 32, 2.7%
Every day: 14, 1.2%
No answer: 139, 11.7%
MODAFINIL:
Never: 76.5%
Rarely: 78, 6.6%
>1x/month: 48, 4.1%
Every day: 9, 0.8%
No answer: 143, 12.1%
TRUE PRISONERS' DILEMMA:
Defect: 341, 28.8%
Cooperate: 316, 26.7%
Not sure: 297, 25.1%
No answer: 229, 19.4%
FREE WILL:
Not confused: 655, 55.4%
Somewhat confused: 296, 25%
Confused: 81, 6.8%
No answer: 151, 12.8%
TORTURE VS. DUST SPECKS
Choose dust specks: 435, 36.8%
Choose torture: 261, 22.1%
Not sure: 225, 19%
Don't understand: 22, 1.9%
No answer: 240, 20.3%
SCHRODINGER EQUATION:
Can't calculate it: 855, 72.3%
Can calculate it: 175, 14.8%
No answer: 153, 12.9%
PRIMARY LANGUAGE:
English: 797, 67.3%
German: 54, 4.5%
French: 13, 1.1%
Finnish: 11, 0.9%
Dutch: 10, 0.9%
Russian: 15, 1.3%
Portuguese: 10, 0.9%
These are all the languages with ten or more speakers, but we also have everything from Marathi to Tibetan. You can see the full table here..
NEWCOMB'S PROBLEM
One-box: 726, 61.4%
Two-box: 78, 6.6%
Not sure: 53, 4.5%
Don't understand: 86, 7.3%
No answer: 240, 20.3%
ENTREPRENEUR:
Don't want to start business: 447, 37.8%
Considering starting business: 334, 28.2%
Planning to start business: 96, 8.1%
Already started business: 112, 9.5%
No answer: 194, 16.4%
ANONYMITY:
Post using real name: 213, 18%
Easy to find real name: 256, 21.6%
Hard to find name, but wouldn't bother me if someone did: 310, 26.2%
Anonymity is very important: 170, 14.4%
No answer: 234, 19.8%
HAVE YOU TAKEN A PREVIOUS LW SURVEY?
No: 559, 47.3%
Yes: 458, 38.7%
No answer: 116, 14%
TROLL TOLL POLICY:
Disapprove: 194, 16.4%
Approve: 178, 15%
Haven't heard of this: 375, 31.7%
No opinion: 249, 21%
No answer: 187, 15.8%
MYERS-BRIGGS
INTJ: 163, 13.8%
INTP: 143, 12.1%
ENTJ: 35, 3%
ENTP: 30, 2.5%
INFP: 26, 2.2%
INFJ: 25. 2.1%
ISTJ: 14, 1.2%
No answer: 715, 60%
This includes all types with greater than 10 people. You can see the full table here.
Part 3: Numerical Data
Except where indicated otherwise, all the numbers below are given in the format:
mean+standard_deviation (25% level, 50% level/median, 75% level) [n = number of data points]
INTELLIGENCE:
IQ (self-reported): 138.7 + 12.7 (130, 138, 145) [n = 382]
SAT (out of 1600): 1485.8 + 105.9 (1439, 1510, 1570) [n = 321]
SAT (out of 2400): 2319.5 + 1433.7 (2155, 2240, 2320)
ACT: 32.7 + 2.3 (31, 33, 34) [n = 207]
IQ (on iqtest.dk): 125.63 + 13.4 (118, 130, 133) [n = 378]
I am going to harp on these numbers because in the past some people have been pretty quick to ridicule this survey's intelligence numbers as completely useless and impossible and so on.
According to IQ Comparison Site, an SAT score of 1485/1600 corresponds to an IQ of about 144. According to Ivy West, an ACT of 33 corresponds to an SAT of 1470 (and thence to IQ of 143).
So if we consider self-report, SAT, ACT, and iqtest.dk as four measures of IQ, these come out to 139, 144, 143, and 126, respectively.
All of these are pretty close except iqtest.dk. I ran a correlation between all of them and found that self-reported IQ is correlated with SAT scores at the 1% level and iqtest.dk at the 5% level, but SAT scores and IQTest.dk are not correlated with each other.
Of all these, I am least likely to trust iqtest.dk. First, it's a random Internet IQ test. Second, it correlates poorly with the other measures. Third, a lot of people have complained in the comments to the survey post that it exhibits some weird behavior.
But iqtest.dk gave us the lowest number! And even it said the average was 125 to 130! So I suggest that we now have pretty good, pretty believable evidence that the average IQ for this site really is somewhere in the 130s, and that self-reported IQ isn't as terrible a measure as one might think.
AGE:
27.8 + 9.2 (22, 26, 31) [n = 1185]
LESS WRONG USE:
Karma: 1078 + 2939.5 (0, 4.5, 136) [n = 1078]
Months on LW: 26.7 + 20.1 (12, 24, 40) [n = 1070]
Minutes/day on LW: 19.05 + 24.1 (5, 10, 20) [n = 1105]
Wiki views/month: 3.6 + 6.3 (0, 1, 5) [n = 984]
Wiki edits/month: 0.1 + 0.8 (0, 0, 0) [n = 984]
PROBABILITIES:
Many Worlds: 51.6 + 31.2 (25, 55, 80) [n = 1005]
Aliens (universe): 74.2 + 32.6 (50, 90, 99) [n = 1090]
Aliens (galaxy): 42.1 + 38 (5, 33, 80) [n = 1081]
Supernatural: 5.9 + 18.6 (0, 0, 1) [n = 1095]
God: 6 + 18.7 (0, 0, 1) [n = 1098]
Religion: 3.8 + 15.5 (0, 0, 0.8) [n = 1113]
Cryonics: 18.5 + 24.8 (2, 8, 25) [n = 1100]
Antiagathics: 25.1 + 28.6 (1, 10, 35) [n = 1094]
Simulation: 25.1 + 29.7 (1, 10, 50) [n = 1039]
Global warming: 79.1 + 25 (75, 90, 97) [n = 1112]
No catastrophic risk: 71.1 + 25.5 (55, 80, 90) [n = 1095]
Space: 20.1 + 27.5 (1, 5, 30) [n = 953]
CALIBRATION:
Year of Bayes' birth: 1767.5 + 109.1 (1710, 1780, 1830) [n = 1105]
Confidence: 33.6 + 23.6 (20, 30, 50) [n= 1082]
MONEY:
Income/year: 50,913 + 60644.6 (12000, 35000, 74750) [n = 644]
Charity/year: 444.1 + 1152.4 (0, 30, 250) [n = 950]
SIAI/CFAR charity/year: 309.3 + 3921 (0, 0, 0) [n = 961]
Aging charity/year: 13 + 184.9 (0, 0, 0) [n = 953]
TIME USE:
Hours online/week: 42.4 + 30 (21, 40, 59) [n = 944]
Hours reading/week: 30.8 + 19.6 (18, 28, 40) [n = 957]
Hours writing/week: 7.9 + 9.8 (2, 5, 10) [n = 951]
POLITICAL COMPASS:
Left/Right: -2.4 + 4 (-5.5, -3.4, -0.3) [n = 476]
Libertarian/Authoritarian: -5 + 2 (-6.2, -5.2, -4)
BIG 5 PERSONALITY TEST:
Big 5 (O): 60.6 + 25.7 (41, 65, 84) [n = 453]
Big 5 (C): 35.2 + 27.5 (10, 30, 58) [n = 453]
Big 5 (E): 30.3 + 26.7 (7, 22, 48) [n = 454]
Big 5 (A): 41 + 28.3 (17, 38, 63) [n = 453]
Big 5 (N): 36.6 + 29 (11, 27, 60) [n = 449]
These scores are in percentiles, so LWers are more Open, but less Conscientious, Agreeable, Extraverted, and Neurotic than average test-takers. Note that people who take online psychometric tests are probably a pretty skewed category already so this tells us nothing. Also, several people got confusing results on this test or found it different than other tests that they took, and I am pretty unsatisfied with it and don't trust the results.
AUTISM QUOTIENT
AQ: 24.1 + 12.2 (17, 24, 30) [n = 367]
This test says the average control subject got 16.4 and 80% of those diagnosed with autism spectrum disorders get 32+ (which of course doesn't tell us what percent of people above 32 have autism...). If we trust them, most LWers are more autistic than average.
CALIBRATION:
Reverend Thomas Bayes was born in 1701. Survey takers were asked to guess this date within 20 years, so anyone who guessed between 1681 and 1721 was recorded as getting a correct answer. The percent of people who answered correctly is recorded below, stratified by the confidence they gave of having guessed correctly and with the number of people at that confidence level.
0-5: 10% [n = 30]
5-15: 14.8% [n = 183]
15-25: 10.3% [n = 242]
25-35: 10.7% [n = 225]
35-45: 11.2% [n = 98]
45-55: 17% [n = 118]
55-65: 20.1% [n = 62]
65-75: 26.4% [n = 34]
75-85: 36.4% [n = 33]
85-95: 60.2% [n = 20]
95-100: 85.7% [n = 23]
Here's a classic calibration chart. The blue line is perfect calibration. The orange line is you guys. And the yellow line is average calibration from an experiment I did with untrained subjects a few years ago (which of course was based on different questions and so not directly comparable).
The results are atrocious; when Less Wrongers are 50% certain, they only have about a 17% chance of being correct. On this problem, at least, they are as bad or worse at avoiding overconfidence bias as the general population.
My hope was that this was the result of a lot of lurkers who don't know what they're doing stumbling upon the survey and making everyone else look bad, so I ran a second analysis. This one used only the numbers of people who had been in the community at least 2 years and accumulated at least 100 karma; this limited my sample size to about 210 people.
I'm not going to post exact results, because I made some minor mistakes which means they're off by a percentage point or two, but the general trend was that they looked exactly like the results above: atrocious. If there is some core of elites who are less biased than the general population, they are well past the 100 karma point and probably too rare to feel confident even detecting at this kind of a sample size.
I really have no idea what went so wrong. Last year's results were pretty good - encouraging, even. I wonder if it's just an especially bad question. Bayesian statistics is pretty new; one would expect Bayes to have been born in rather more modern times. It's also possible that I've handled the statistics wrong on this one; I wouldn't mind someone double-checking my work.
Or we could just be really horrible. If we haven't even learned to avoid the one bias that we can measure super well and which is most susceptible to training, what are we even doing here? Some remedial time at PredictionBook might be in order.
HYPOTHESIS TESTING:
I tested a very few of the possible hypothesis that were proposed in the survey design threads.
Are people who understand quantum mechanics are more likely to believe in Many Worlds? We perform a t-test, checking whether one's probability of the MWI being true depends on whether or not one can solve the Schrodinger Equation. People who could solve the equation had on average a 54.3% probability of MWI, compared to 51.3% in those who could not. The p-value is 0.26; there is a 26% probability this occurs by chance. Therefore, we fail to establish that people's probability of MWI varies with understanding of quantum mechanics.
Are there any interesting biological correlates of IQ? We run a correlation between self-reported IQ, height, maternal age, and paternal age. The correlations are in the expected direction but not significant.
Are there differences in the ways men and women interact with the community? I had sort of vaguely gotten the impression that women were proportionally younger, newer to the community, and more likely to be referred via HPMOR. The average age of women on LW is 27.6 compared to 27.7 for men; obviously this difference is not significant. 14% of the people referred via HPMOR were women compared to about 10% of the community at large, but this difference is pretty minor. Women were on average newer to the community - 21 months vs. 39 for men - but to my surprise a t-test was unable to declare this significant. Maybe I'm doing it wrong?
Does the amount of time spent in the community affect one's beliefs in the same way as in previous surveys? I ran some correlations and found that it does. People who have been around longer continue to be more likely to believe in MWI, less likely to believe in aliens in the universe (though not in our galaxy), and less likely to believe in God (though not religion). There was no effect on cryonics this time.
In addition, the classic correlations between different beliefs continue to hold true. There is an obvious cluster of God, religion, and the supernatural. There's also a scifi cluster of cryonics, antiagathics, MWI, aliens, and the Simulation Hypothesis, and catastrophic risk (this also seems to include global warming, for some reason).
Are there any differences between men and women in regards to their belief in these clusters? We run a t-test between men and women. Men and women have about the same probability of God (men: 5.9, women: 6.2, p = .86) and similar results for the rest of the religion cluster, but men have much higher beliefs in for example antiagathics (men 24.3, women: 10.5, p < .001) and the rest of the scifi cluster.
DESCRIPTIONS OF LESS WRONG
Survey users were asked to submit a description of Less Wrong in 140 characters or less. I'm not going to post all of them, but here is a representative sample:
- "Probably the most sensible philosophical resource avaialble."
- "Contains the great Sequences, some of Luke's posts, and very little else."
- "The currently most interesting site I found ont the net."
- "EY cult"
- "How to think correctly, precisely, and efficiently."
- "HN for even bigger nerds."
- "Social skills philosophy and AI theorists on the same site, not noticing each other."
- "Cool place. Any others like it?"
- "How to avoid predictable pitfalls in human psychology, and understand hard things well: The Website."
- "A bunch of people trying to make sense of the wold through their own lens, which happens to be one of calculation and rigor"
- "Nice."
- "A font of brilliant and unconventional wisdom."
- "One of the few sane places on Earth."
- "Robot god apocalypse cult spinoff from Harry Potter."
- "A place to converse with intelligent, reasonably open-minded people."
- "Callahan's Crosstime Saloon"
- "Amazing rational transhumanist calming addicting Super Reddit"
- "Still wrong"
- "A forum for helping to train people to be more rational"
- "A very bright community interested in amateur ethical philosophy, mathematics, and decision theory."
- "Dying. Social games and bullshit now >50% of LW content."
- "The good kind of strange, addictive, so much to read!"
- "Part genuinely useful, part mental masturbation."
- "Mostly very bright and starry-eyed adults who never quite grew out of their science-fiction addiction as adolescents."
- "Less Wrong: Saving the world with MIND POWERS!"
- "Perfectly patternmatches the 'young-people-with-all-the-answers' cliche"
- "Rationalist community dedicated to self-improvement."
- "Sperglord hipsters pretending that being a sperglord hipster is cool." (this person's Autism Quotient was two points higher than LW average, by the way)
- "An interesting perspective and valuable database of mental techniques."
- "A website with kernels of information hidden among aspy nonsense."
- "Exclusive, elitist, interesting, potentially useful, personal depression trigger."
- "A group blog about rationality and related topics. Tends to be overzealous about cryogenics and other pet ideas of Eliezer Yudkowsky."
- "Things to read to make you think better."
- "Excellent rationality. New-age self-help. Worrying groupthink."
- "Not a cult at all."
- "A cult."
- "The new thing for people who would have been Randian Objectivists 30 years ago."
- "Fascinating, well-started, risking bloat and failure modes, best as archive."
- "A fun, insightful discussion of probability theory and cognition."
- "More interesting than useful."
- "The most productive and accessible mind-fuckery on the Internet."
- "A blog for rationality, cognitive bias, futurism, and the Singularity."
- "Robo-Protestants attempting natural theology."
- "Orderly quagmire of tantalizing ideas drawn from disagreeable priors."
- "Analyze everything. And I do mean everything. Including analysis. Especially analysis. And analysis of analysis."
- "Very interesting and sometimes useful."
- "Where people discuss and try to implement ways that humans can make their values, actions, and beliefs more internally consistent."
- "Eliezer Yudkowsky personality cult."
- "It's like the Mormons would be if everyone were an atheist and good at math and didn't abstain from substances."
- "Seems wacky at first, but gradually begins to seem normal."
- "A varied group of people interested in philosophy with high Openness and a methodical yet amateur approach."
- "Less Wrong is where human algorithms go to debug themselves."
- "They're kind of like a cult, but that doesn't make them wrong."
- "A community blog devoted to nerds who think they're smarter than everyone else."
- "90% sane! A new record!"
- "The Sequences are great. LW now slowly degenerating to just another science forum."
- "The meetup groups are where it's at, it seems to me. I reserve judgment till I attend one."
- "All I really know about it is this long survey I took."
- "The royal road of rationality."
- "Technically correct: The best kind of correct!"
- "Full of angry privilege."
- "A sinister instrument of billionaire Peter Thiel."
- "Dangerous apocalypse cult bent on the systematic erasure of traditional values and culture by any means necessary."
- "Often interesting, but I never feel at home."
- "One of the few places I truly feel at home, knowing that there are more people like me."
- "Currently the best internet source of information-dense material regarding cog sci, debiasing, and existential risk."
- "Prolific and erudite writing on practical techniques to enhance the effectiveness of our reason."
- "An embarrassing Internet community formed around some genuinely great blog writings."
- "I bookmarked it a while ago and completely forgot what it is about. I am taking the survey to while away my insomnia."
- "A somewhat intimidating but really interesting website that helps refine rational thinking."
- "A great collection of ways to avoid systematic bias and come to true and useful conclusions."
- "Obnoxious self-serving, foolish trolling dehumanizing pseudointellectualism, aesthetically bankrupt."
- "The cutting edge of human rationality."
- "A purveyor of exceedingly long surveys."
PUBLIC RELEASE
That last commenter was right. This survey had vastly more data than any previous incarnation; although there are many more analyses I would like to run I am pretty exhausted and I know people are anxious for the results. I'm going to let CFAR analyze and report on their questions, but the rest should be a community effort. So I'm releasing the survey to everyone in the hopes of getting more information out of it. If you find something interesting you can either post it in the comments or start a new thread somewhere.
The data I'm providing is the raw data EXCEPT:
- I deleted a few categories that I removed halfway through the survey for various reasons
- I deleted 9 entries that were duplicates of other entries, ie someone pressed 'submit' twice.
- I deleted the timestamp, which would have made people extra-identifiable, and sorted people by their CFAR random number to remove time order information.
- I removed one person whose information all came out as weird symbols.
- I numeralized some of the non-numeric data, especially on the number of months in community question. This is not the version I cleaned up fully, so you will get to experience some of the same pleasure I did working with the rest.
- I deleted 117 people who either didn't answer the privacy question or who asked me to keep them anonymous, leaving 1067 people.
Here it is: Data in .csv format , Data in Excel format