= 27d567bc2d48d9f40e9ccc20ea370b15)
Viliam_Bur comments on 2011 Less Wrong Census / Survey - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (694)
For the gender question it may make sense to have a generic "other" option. The monogamous/polygamous question should also maybe have a no preference option also.
Edit: And finished.
I think it is generally good to avoid "other" options as much as possible.
There are a few biases related to filling questionnaires. For example, many psychological tests ask you the same question twice, in opposite direction. (Question #13 "Do you think Singularity will happen?" Question #74: "Do you think Singularity will never happen?") This is because some people use heuristics "when unsure, say yes" and some other people use heuristics "when unsure, say no". So when you get two "yes" answers or two "no" answers to opposite forms of the question, you know that the person did not really answer the question.
Another bias is that when given three choices "yes", "no" and "maybe", some people will mostly choose "yes" or "no" answers, while others will prefer "maybe" answers. It does not necesarily mean that they have different opinions on the subject. It may possibly mean that they both think "yes, with 80% certainty", but for one of them this means "yes", and for the other one this means "maybe". So instead of measuring their opinions on the subject, you are measuring their opinions on how much certainty is necessary to answer "yes" or "no" in the questionnaire.
Perhaps in some situations the "other" option is necessary, because for some people none of the available options is good even as a very rough approximation. But I think it should be used very carefully, because it encourages the "I am a special snowflake" bias. For example, if someone has no sexual feelings at all, then of course the "monogamy or polygamy" question does not make sense for them. But if it is "I like the idea of being in love with one special person, but I also like the idea of having sexual access to many attractive people" then IMHO this attitude does not deserve a separate category and can be rounded towards one of the choices.
There are a number of types of snowflakes.
If you decide in advance that you aren't going to listen to anyone who doesn't fit your categories, you might be missing something.
You can have:
a) a survey, where everyone's individual differences are rounded into a few given categories;
b) a collection of blog articles, where everyone describes themselves exactly as they desire; or
c) a kind of survey, where some participants send a blog article instead of data.
Both (a) and (b) are valid options, each of them serves a different purpose. I would prefer to avoid (c), because it tries to do both things at the same time, and accomplishes neither. An answer "other" sometimes means "no answer is even approximately correct", but sometimes is just means "I prefer to send you a blog article instead of survey data". The first objection is valid, and is IMHO equivalent to simply not answering that question. The second objection seems more like refusing the idea of statistics. Statistics does not mean that people who gave the same answer are all perfectly alike, but ignoring the minor differences allows us to see the forest instead of the trees.
I guess the "special snowflake bias" is officially called "narcissism of small differences". The psychological foundation is that we have a need of identity, which is threatened by similar things, not different ones. So when something is similar to us, but not the same, we exaggerate the difference and downplay the similarity. From outside view we are probably less different than from inside view.
That last varies-- sometimes people are exaggerating differences which are pretty meaningless. Sometimes the people setting up the classifications actually have an incomplete picture of the existing categories.
There's an established way of correcting for this in market research (and other fields): coding. Let's say you have the following list:
Then you have someone go through all the typed-in responses, and when someone types in "special snowflake utilitarian" you code that as a 1 rather than a 4.
This is also done for completely open-end responses. Sometimes something like "additional comments" will on the back end look like:
Problem is that survey results will be treated as if everyone had exact answers, as opposed to picking the least terrible approximation. (I do have a known preference, dammit! It's just the subject of Big Debate whether it counts as mono or as poly.)
So how you do decide which options merit inclusion? Which snowflakes are special enough--or, I suppose, mundane enough? And what's the harm in counting how many snowflakes aren't, even if you don't ask them exactly what type they are?