It's that time of year again. Actually, a little earlier than that time of year, but I'm pushing it ahead a little to match when Ozy and I expect to have more free time to process the results.
The first draft of the 2014 Less Wrong Census/Survey is complete (see 2013 results here) .
You can see the survey below if you promise not to try to take the survey because it's not done yet and this is just an example!
2014 Less Wrong Census/Survey Draft
I want two things from you.
First, please critique this draft (it's much the same as last year's). Tell me if any questions are unclear, misleading, offensive, confusing, or stupid. Tell me if the survey is so unbearably long that you would never possibly take it. Tell me if anything needs to be rephrased.
Second, I am willing to include any question you want in the Super Extra Bonus Questions section, as long as it is not offensive, super-long-and-involved, or really dumb. Please post any questions you want there. Please be specific - not "Ask something about taxes" but give the exact question you want me to ask as well as all answer choices.
Try not to add more than a few questions per person, unless you're sure yours are really interesting. Please also don't add any questions that aren't very easily sort-able by a computer program like SPSS unless you can commit to sorting the answers yourself.
I will probably post the survey to Main and officially open it for responses sometime early next week.
- I suggest a basilisk question, as usual.
- I'd also like OCEAN/Big Five restored. The results from last time weren't very definitive and more data would be helpful.
- Given the recent SSC debate on how effective MIRI is, it might be interesting to ask something about opinions of MIRI's effectiveness in pursuing its mission: just asking how much people donated is insufficient since one might donate faute de mieux, and IIRC the modal donation is $0 in the first place.
- Perhaps the AI projection questions could be replaced by the same question-set used in "Future progress in artificial intelligence: A poll among experts", VC Müller, N Bostrom? Buys one comparability with their survey responses; it would be very interesting and intriguing if LWers turned out to be less extreme.
I would personally be interested in more detailed drug use questions. Examplae. In the Past year have you taken:
-Modafinil
-Amphetamine (of any sort including Adderall)
-Heroine/Other Opiates (not prescribed by a doctor as a painkiller) -Marijuanna
-Hallucinogens (LSD/DMT/Psychobillin)
-Testosterone/HGH (if you took testosterone because you are transexual do not click yes)
Maybe there is a better list of drugs?
I had this last time, and several people told me to take it off because it was bad to make people admit to illegal activities.
Also, for complicated reasons I can't do "Check as many as apply" questions, so this would take forever.
The question about "Country" should clarify whether you are asking about nationality or residence.
The "currently looking for more relationship partners" feels to me ambiguous -- what if someone is open to having more relationship partners, but not actually looking for more? Perhaps make this a three option issue "actively looking" vs "open to having more relationship partners but not actively looking for more" vs "not open to having more relationship partners"?
I'd add: "Do you use/have you used a Spaced Repetition program, like Anki or Supermemo?"
- No
- Yes, regularly
- I tried it, but stopped
In the politics I'd add a "how strongly do you care about this political affiliation" (scaled on 1-5, from "not at all" to "it's an important part of my identity")
In the politics I'd add a "how strongly do you care about this political affiliation" (scaled on 1-5, from "not at all" to "it's an important part of my identity")
A slightly longer option is to ask people to rate every political affiliation from 1 to 5. This lets us identify clusters better, as well as total political identification (if someone gives 1s to everything, that's the 'apolitical' option).
A couple (more) questions I'd find interesting:
How many times do you exercise per week, on average?
How many nonfiction books do you read per month, on average?
How knowledgeable would you consider yourself in the following fields? (on a scale of 0 to 5, where 3 is about "studied it at university" and 5 "I'm a publically recognized expert")
Psychology
Economics
Artificial Intelligence
Statistics
Formidable work. Given recent discussions, might it be worth adding "vegan" to "vegetarian"? (And perhaps even "pescetarian" or "flexitarian" but I suppose one can get lost in small distinctions.)
I'd propose the following three:
- vegan
- vegetarian
- reduced meat intake
The latter, 'reduced meat intake' is intended to represent pescetarian, flexitarian, and meat reduction all in one.
I'd been interested in seeing how many people participate in systematic self-improvement efforts outside the LW/CFAR space. Some examples in rough increasing order of weirdness: diet, fitness, psychiastric help, life coaching etc., social skills programs of various kinds, nootropics. This covers a lot of ground and there's probably some options I haven't thought of, but these two questions would be a good start:
- Do you follow a limited or unusual diet for health (not ethical) reasons?
(this would probably need "I tried and it didn't work" and "I tried but found it too hard" options. We'd probably also want some way of distinguishing between "I'm allergic to onions" and "I kind of try not to eat red meat" and "I'm full-blown hardcore paleo".)
- How many hours a week do you spend exercising, on average?
(for example, 0 if you are single, 1 if you are in a monogamous relationship, higher numbers for polyamorous relationships)
This implies that you are only poly if you're in more than 1 relationship. I suggest simply
(for example, 0 if you are single, 1 if you are in a monogamous relationship)
because
- If you're poly, the idea of counting relationships will be familiar to you already; it's only those who aren't who might need it set out explicitly
- I hate saying "you have to add more text here to cater to my itch" - if I can fix what I perceive to be a problem by removing text, I much prefer that.
I feel like using Scandanavia as an example of "socialism" is not really accurate - they're capitalist welfare states with slightly higher taxes than other capitalist welfare states.
That definition of "socialism" conflicts with the platforms of parties caling themselves "socialist", which is probably what most people using that label have in mind.
For relationship status, a polyamorous person can be married and in a relationship at the same time, which is a problem. Similarly, someone can be living with their partner/spouse and additional roommates. Also, "Liberal" in the Political section should probably be renamed to "Progressive", to avoid collisions with how "liberal" is used in Europe and in political philosophy.
Please change the "referred by a link on another blog" option to "referred by a link on another blog or website". It's been bugging me for years.
Again, your relationship style question conflates very different clusters. There is a big difference between traditional monogamy and serial monogamy, or between the ideal of polyamory preached in rationalist circles and the harem type of polyamory (polygyny, mistresses, etc...) which is traditional for a sufficiently high-status man to have. The obvious way to solve this is to split the "prefer monogamous" and "prefer polyamorous" answers into two answers each, each of which describes the relevant clusters in a short sentence similar to the answer choices in the political question.
You are asking too many calibration questions for me. I might answer 1, 2, or, at a stretch, 3. 10 questions is gonna cause me to leave that whole section blank.
Likewise, I don't think I'm gonna answer the probabilities section unless you switch to radio buttons, like you use in the extra credit political section.
Per your own advice, you may want to add a completely implausible question to the survey so you can measure ou...
Per your own advice, you may want to add a completely implausible question to the survey so you can measure our Lizardman's constant and subtract it from the other results.
+1
- Neither of "...and currently looking for more relationship partners" or "...and not currently looking for more relationship partners" seem to capture what I feel is a common poly sentiment, "...and I'll figure out what I want with new people as I get to know them".
- Neither of "Atheist but spiritual" and "Atheist but not spiritual" seem to capture "Atheist who doesn't know what 'spiritual' means in this context and therefore doesn't want to come down either for or against it, because there are both int
(hint: if you've been here since the start of the community in November 2007, put 6)
This should presumably be 7 now.
Consider asking people in what year they joined the community? So if you've been here since the start, put 2007.
If there's no good way to ask about specific supplements in the survey, would anyone mind if I raised it as a question in Discussion?
How about a survey question about how much people care about their LW karma?
For Super Extra Bonus Questions: (feel free to modify the answer choices)
With which of these metaethical positions do you most identify?
- Non-cognitivism: Moral statements don't express propositions and can neither be true nor false. "Murder is wrong" means something like "Boo murder!".
- Error theory: Moral statements have a truth-value, but attempt to describe features of the world that don't exist. "Murder is wrong" and "Murder is right" are both false statements because moral rightness and wrongness aren't features
Thanks so much for running this again
Comments
-Items should either be randomized or if sorted, they should be sorted based on last years prevelency. For example in politics libertarian is the top listed item despite not being alphabetically first or being a majorty / plurality view.
-The religious denomination shouldn't ask atheists to skip the question. One is about what you believe the other about what you do. Plenty of atheists fast on Yom Kippur or go to church on Christmas.
-Less wrong use should have two items on comments that differentiate by frequen...
If this survey generates interesting psychometric research, someone might try to get a journal article out of it. If so, we will need your explicit consent to have (an aggregate of) your anonymous data published.
Someone? Who is someone? Honestly, I'm curious, because I can't think of who these someones would be. What research psychologists who aren't already members of Less Wrong pay attention to it? I doubt it would be Scott using it to publish something(?) Maybe he shares the results on Slate Star Codex, and a psychiatrist friend of his considers Les
In the mental health category, I'd love to see (adult) ADHD there as well. I'm less directly interested in substance abuse disorder and learning disabilities (in the US sense) / non-autism developmental disabilities, but those would be interesting additions too.
Some thoughts from my notes when I did the LWSH survey. I mostly plagiarized it from you, so they should be relevant:
Consider adding an option under work and/or profession for "Homemaker." We have at least one.
Consider an option under the relationship goals section for "not looking, but open to the possibility."
There used to be a whole section on akrasia. I note that it is now missing, but it had questions about use of medication and supplements. Those questions didn't specify whether the meds were taken specifically to combat akrasia....
How would you describe your opinion on immigration?
Immigration into one's own country (as for the following two questions) or in general?
- Given that you already ask the Sex question, the Gender question can have just three answers “Man”, “Woman”, and “Neither/Other”.
- As suggested by people on SSC, you might want to add a question “Is your gender an important part of your identity?” or something, to find out how many of us are cis-by-default.
- I'd have three answers to Relationship Goals, “actively looking”, “open but not actively looking”, and “not open”.
- Split the “no” answer to Children into “not now” and “never”.
- Selfishly, I'm glad you didn't split physics in the Profession question into
Anything I do with gender and sex is going to have lots of people yell at me. But if I keep it the same, it will be the same people as last year and I won't make new enemies.
Is anyone interested in asking whether people are cosmopolitan?
How about natalist versus antinatalist?
[Added] I don't care about this that much, but here's a question text for cosmopolitanism: "In prioritizing political or charitable causes, would you weight the interests of a person of another nationality equally with those of a fellow citizen?" (Yes or No) A link to Chris's post might also help.
Not exactly a question per se, but I remember Yvain complaining about invalid entries, such as words in the number field etc. etc.
This sounds like something that simple Regular Expressions could speed up. Perhaps we could ask him what his current survey workflow is and try to see where they can fit?
Although this advice seems a bit TOO obvious not to have been mentioned before. Apologies if so.
The "with family" option in the "living with" question is ambiguous for those of us with children. I suggest changing it to "with parents or guardians[1]", changing the partner/spouse option to "with partner/spouse (and children if applicable)", and adding an "other" option for less traditional living arrangements.
Questions in the mental health section are inconsistent about whether they're referring to whether you have ever suffered from a condition ("have you ever been diagnosed...") or whether ...
I'll write down a long list of questions. I think they are useful. If you don't like them all because they are too much I would certainly like to keep data about spaced repetition software usage. The question from last year was good.
I would specifically like to have the bolted questions included. With them we could cut the question that asks for self identification as vegetarian.
Given the absence of perfect information our justice system has to convict a few innocent people. Where the optimum of the ratio between guilty going free and innocent getting se...
- Order the countries by their frequency in previous surveys.
- As ever, the political labels used are misleading. The Scandinavian countries are not socialist, and the UK labour party are in many ways extremely illiberal.
- You might want to break the cryonics question up a little more - some people are 'just finishing up the paperwork' for a very long time.
- The immigration question is presumably actually asking about immigration controls, not immigration itself.
You have the "Slate Star Codex" question under the "privacy" section. You may want to retitle the section to something like "top matter".
I'd like an "apolitical" choice for my political views. Especially here, there's likely a significant slice of the population who choose to not have any politics in the manner of Paul Graham, and thus equally don't identify at all, not even a little bit, with any of the given political options.
"If you were referred here by another blog—" "another" is needless.
Con...
Formatting issues:
- The title "Part Eight: Slightly More Complicated Questions" appears twice.
Question requests:
- Ability to solve the Schrodinger equation for the hydrogen atom.
- OCEAN personality test results
- Split "no" option in meetups into "no, because there are no meetups near where I live" and "no, there may be meetups near where I live but I don't want to go to them"
Other comments:
- I like the multiple calibration questions
Ability to solve the Schrodinger equation for the hydrogen atom.
In case you care about that in order to know which respondents know what they're talking about when answering the MWI question, that's a very poor choice (as I mentioned two years ago IIRC). It basically mostly only checks whether people took QM classes (in many of which interpretational issues are discussed hardly at all) and can remember the tricks to solve second-order differential equations in spherical coordinates. Asking whether people can prove Bell's theorem would be a much better choice. (You should weigh Scott Aaronson's opinion about MWI over mine even though I'm a physicist and he isn't.) Having read How the Hippies Saved Physics, I'd guess that if anything ability to solve the SE for the H atom would anti-correlate with trustworthiness about interpretations of QM when controlling for work status, profession and degree.
OCEAN personality test results
Seconded.
What is your preferred relationship style?
No explicit option for asexual and/or aromantic? (Or whatever the term would be for preferring not to be in a relationship, as opposed to not having a preference.)
Please answer on a scale from 0% (definitely false) to 100% (definitely true). For your convenience, 0% will be interpreted as "epsilon" and 100% as "100 minus epsilon".
For estimated probabilities that are greater than 99%, or less than 1%, how many digits of accuracy do you wish the answer to be? Just jump straight from 99% to ...
way fewer people use LW now, it would be interesting to track immigration and emmigration. Add some questions to see what time period people report their peak use being / what people think changed ?
You should add a question that asks what model of population ethics they support (Additive utility, Average utility, Other, Non-utilitarian, Don't know).
Question suggestions:
Only answer if you think the chance of a singularity in the 21st century is over 1%. Do you think affecting the singularity's outcome is tractable enough to be worth any of your time or money?
Are you a narcissist? (rationale)
(This is only for people who had an SAT score out of 1600) Did you take the SAT after (or during) April 1995?
May want to add Slate Star Codex as an exception to the referrals question.
Time in community question needs to be updated to 7 years for the start of the community.
Might be worth it to specify aggregate Karma if you have multiple accounts. (This is an account that I started using after I decided I no longer wanted to use my real name. I mostly lurk anyway, though.)
It would be worth it to add a "no meetups in my area" option to the meetups question.
The header for part eight is listed twice.
I've been reading the thread, and it appears there are some categories of questions Less Wrong wants explored in more detail, with more questions. These subjects include:
- rates of vegetarianism/paleo/alt-diets
- what drugs do you take without a prescription, including caffeine/nicotine/alcohol, nootropics, and illicit recreational drugs.
- a wide diversity of philosophical moral/meta-ethical positions
- opinions of certain policies
- wider variety of sexual/romantic/gender orientation options.
This is all very interesting. However, with upvotes, below all these ...
Suggestions for Questions Specific to the Center for Applied Rationality
Maybe take the opportunity to ask more questions about CFAR, and ask the CFAR staff you know if they want questions answered, because they love data. As a CFAR workshop alumnus, I'm interested in seeing where that community is headed. Here are some suggestions:
If you're not one of the CFAR alumni, what's your estimate that you will attend one of their workshops in the next [six months, one year, two year, insert timescale here]?
If you haven't attended a CFAR workshop, do you ever i
All the time, people put DO NOT CITE on their papers, but that digit ratio web page had a link, explaining why you shouldn't cite: because it's just a copy of wikipedia.
Also, handfacts.com looks really sketchy to me. Not only is it no longer handfacts.com nor e-hand.com, but now eatonhand.com, but it doesn't have instructions on measuring digit ratio.
Typos in digit ratio question: "Use a ruler of calipers", "The result is 2D:4D" (should that be 2D/4D)? This question could probably use an image.
1a) If you're planning on comparing IQ scores to SAT scores, you should include age at which the respondent took the SAT. I know of at least one international gifted program, run by Johns Hopkins University, which administers the SAT to 12-year-olds.
1b) Scores on the different sections of the SAT might be interesting. E.g. SAT Score out of 1600; SAT-Verbal score out of 800; SAT-Math score out of 800 and SAT Score out of 2400; SAT-Reading score out of 800; SAT-Writing score out of 800; SAT-Math score out of 800.
2) A mere typo: "Do you attend Less Wrong...
First of all, the survey is excellent as it is. So I, too, merely suggest an addition.
I'd like to see added something about teaching / spreading rationality / proselytizing. My first idea would be something like: "Do you attempt to make others think more rationally?" with options on a spectrum from "no" and "I generally try to be the voice of reason in discussions" to something like "Many know me as an advocate for rationality". Not sure about how to adress an online/offline divide.
"None of your family / friends / acquaintances will suffer from any form of cancer anymore, nor will any of their descendants! How many years of your remaining lifespan would you be willing to give up in exchange for the whole of humanity enjoying the same protection, given your sacrifice would remain anonymous? (If you provide a large number, no life extension would be entailed.)" -- (Requires numerical answer: number of years).
How about:
What country is the most successful at providing a good quality of life for its citizens?
Also:
What historical society emphasized the moral virtues with which you most strongly agree?
Extra Credit: Politics, Question about the Great Stagnation: the link is too large Screenshot
To the Super Bonus Questions, could you add:
Is there a number which you have considered your "favorite number" for some period of time (i.e., from before you read this question)? If so, what is it?
Maybe you could split “Yes” in the Blood question into “Yes, in the last 12 months” and “Yes, longer ago”.
Before the professional intelligence measurement question, please add something like:
Please ESTIMATE your IQ, using a standard average of 100 and stdev of 15.
It might make sense to add on something like:
You may want to check a normal distribution calculator (such as http://stattrek.com/online-calculator/normal.aspx ) to ensure you aren't overestimating.
Or, alternatively, we could get at the same sort of question by asking this:
...Please estimate how many people would have to be in a group of adults, chosen uniformly at random, for you to expect the
Some questions I'd personally put in if I was writing this survey (you don't need to put these in if you don't want to, just suggestions):
- To what extent have your beliefs about the world and your reasoning processes changed since you became a member of this community?
- To what extent do you think that MIRI's mission is a worthy one (i.e. do you think that AI is a major existential risk, do you think it's important to ensure the development of friendly AI, do you think that MIRI is capable of advancing that goal, etc.)
- To what extend do you identify with
Why not use the Kinsey sex scale for sexual orientation? (really depends on what you want to gather out of this information)
A split question of whether your children are biologically related to you or just children you are raising.
Allow multiple options (checkboxes) for work-status, and profession.
I have seen "education level" as highest level completed; but it doesn't take into account highest level incomplete. so where I might be part of a bachelors, I could not say so because I have not yet completed it. Perhaps a level of "highest edu...
P(God)
This question bugged me from last year's survey, but I'm not sure if my objections are valid. Consider a subset of this question: "What is the probability that Bayesian mathematical reasoning was created by a supranatural entity?" It seems... outside the domain, a little bit? Like a question, "What is the probability that Bayesian mathematics is true?"- what would it mean for this to be false? Which is to say, I don't know whether probability estimates are capable of a certain degree of recursive self-analysis, and asking about the origins of Bayesianism may be in that class.
any question you want in the Super Extra Bonus Questions section
Do you like the cartoon, "My Little Pony: Friendship is Magic"?
- Yes, and I consider myself a brony/pegasister
- Yes
- No
- No opinion / unsure
(And to compare that with a fandom that seems to be less represented amongst LWers:)
Are you a furry fan? (Reference link: http://en.wikifur.com/wiki/Furry )
- Yes
- No
- No opinion / unsure
I request that you add an intensity of importance question for each item on the politics section. I might think that abortion is terrible but not at all politically important, for example, and both of those seem like worthwhile information.
The survey is long enough.that I'm unlikely to put genuine thought into each of the questions. If none of the answers us closer enough, I'm likely to skip the question. I'm likely to pick numbers it off thin air on any questions asking for a probability. I'm unlikely to actually dig up my SAT scores etc and will guess from memory. I will probably skip half of the bonus questions that Seth uninteresting because there are so many of them.
There are probably other people who are willing to run through the survey to help out but will spend a fixed and not too large amount of time on it. This is likely to introduce some bias and inaccuracy of answers given its current size. Just noting this as a set of trade offs to keep in mind.
If it's not yet too late, I should like to see:
How often do you use a physical library? Often Rarely Never Online Resources Only
I approve! I think you should put in High school tests in regards to other countries (maybe at least the UK and Australia). It would also be helpful to track GRE scores and similar graduated standardized testing. I think it would be also interesting to see what people we have on the Myers-Briggs testing as well as an optional question. I think that would be interesting to look at on the data side.
In the Religious Denomination question, "If atheist, please skip this question." should be replaced with "If non-religious, please skip this question."
Maybe even remove this sentence and add "non-religious" or "not applicable" as an option.
I think this is a very important change. I think there are many people who identify as Jewish Atheist, Buddhist Atheist, or Unitarian Universalist Atheist, (and maybe some others) and right now you are leaving it up to them to choose how to interpret the question. No information is lost by implementing this change, as there was already a question about theism.
Am I doing the digit thingy right? I scanned my hands and used MS Paint to make a line through each finger, then divided the number of pixels in each line.
Mental Health:
The questions are currently mixing point and lifetime prevalence: "have you ever been diagnosed" with "I believe I have it".
Can we give social anxiety its own question?
A random comment.
This is the first time I've seen "anti-agathics". Based on what I know of biblical Greek, I read this as something that would be like "anti-good". If I had been in charge of making up an anti-aging drug, I would have called it something like anti-presbycs (maybe that wasn't chosen because it looks too much like "presbyterian"? Presbyterian does derive from the world meaning "elder"...).
This isn't a request to change the wording if that's what people who will be taking the survey are familiar with BTW...
Choice suggestions:
Some ideal world would use the empirically valid Eysenck model of left vs. right and authoritarian vs. libertarian for the political section. Oh wait, you basically did. Good job.
Complex Affiliation choice - Inscrutably Idiosyncratic
Autism Spectrum choice - I don't know but my Autism Spectrum Quotient was [blank]
Question suggestion 1:
Only answer this if you think the chance of a singularity before 2100 is over 1%. Do you feel that influencing the singularity's outcome is tractable enough to be worth your time or money?
Question sugg...
I would like that the question for previous surveys asks for the private key that was used in the last survey if the person remembers.
Bring back the Unreasonably Long Questions section (the one with the personality tests etc.), and maybe add the BSRI to it.
Maybe bring back mother's age and father's age in the Super Bonus Questions.
What about a question about PUAs in Extra Credit: Politics?
Is the "Birth Month" bonus question just to sort people arbitrarily into groups to do statistics, or to find correlations between birth month and other traits? If the latter, since the causal mechanism is almost certainly seasonal weather, the question should ask directly for seasonal weather at birth to avoid South Hemisphere confounders.
I would really like to see these questions in the survey:
For the questions:
- Give the time from your birth to your death in subjective years (so years where you are cryonically preserved don't count)
- Give the estimate where you are 50% sure it would be less than your answer and 50% sure it would be more than your answer)
The questions are:
- 1) How long do you think you will live?
- 2) If your only way to die was by really wanting to die, when do you think you would die...
- a) if you could control your aging process but the world would be otherwise unchanged?
Don't you want devoted followers?
Who leave their families for you
Give their money to you
Give their bodies to you
Give up their lives for you
Consider you God, and will kill for you
Don't you want to become a cult leader?
Since the death of God there has been a vacancy open
You can fill that void, here is how
- I'd like to split the Gender:Other option into "I identify as a gender not listed here" and "I don't identify as any particular gender", as I'm curious how many other people have the latter view.
- "roommates" is a bit of an americanism; I might phrase that as "with other people". I'm kind of curious about the split between "with friends" and "with people I didn't otherwise know", but I can't think of anything terribly useful to do with that.
- I'd like a referral option for "Referred by other r
That's how it must be. The question is about which you most identify with. "None of the above" would be confusing in the context of that question.
On the two-dimensional political axis: https://en.wikipedia.org/wiki/Political_compass
The options given are:
- central right (anti-state capitalist)
- moderate right (state-ambivalent capitalist)
- upper-moderate-right (regulationist capitalist)
- Worded as moderate left, but presented as upper-center (Scandinavian welfare states)
- Upper-left (state communist, USSR)
This leaves a rather large portion of the spectrum totally unaccounted for and is going to artificially inflate the neighboring elements.
Adding the following items would fix this:
- Upper-right (Third
Before the professional intelligence measurement question, please add something like:
It might make sense to add on something like:
Or, alternatively, we could get at the same sort of question by asking this:
But it's not obvious to me that this is a thing people would be good at estimating.
That depends a lot on the group of adults you use as reference. World population? The population of the country in which you are living? City? Your facebook friends?
I think IQ is much better because there are objective standards. "Smart" is also a word that different people interpret slightly differently. IQ is much more precise.