I downvoted the article not because I think it is bad, per se -- even though you're other-optimizing -- but because I want to see less of this genre of writing on LW in general.

Just to be pedantic: Enjoying being wrong probably not good.

Enjoying having been wrong, and now being (potentially) less wrong is good.

But the latter doesn't make as good a title :D.

death doesn't scare me nearly as much as old age

O_O

This closely pattern-matches ableist attitudes like "$disability is worse than death!", where people with $disability shrug and say "Enh, we're good. We can't do $thing, but we can't be knitting prodigies either - and neither can you, and you're not whining about it all that much."

Admittedly, losing abilities is no fun, and mental abilities are genuinely good and important (though how much is not that clear - people with Down's are happier), and aging then dying is strictly worse than just dying... but still, scarier than death?

It's definitely a good idea to do this.

But the way you've set about doing it isn't going to produce any worthwhile data.

I'm no expert on branding and market research, but I'm pretty sure that the best practice in the field isn't having conversations with 9 non-random strangers in a lift (asking different leading questions each time) then bunging it in Google Docs and getting other people to add more haphazard data in the hope that someone will make a website that sorts it all out.

First you need to define the question you're asking. Exactly which sub-population are you interested in? You start off asking about "the average person"'s attitude to rationality, suggesting that maybe you want to gauge attitudes across the whole (US?) population. But then you decide that the 60+ man is "outside our demographic bracket", although your 70+ grandmother apparently isn't.

Either way, the set of [people who work in your office building plus your grandmother] might not constitute a representative sample of the population of the USA, let alone everyone in the world. Getting people who frequent Less Wrong to ask people they cross paths with isn't going to be a representative sample of all people - you can see that, right?

The most efficient way to answer your question is likely to be piggybacking on existing polling organisations. Now, it's probably true that corporate marketing/branding "researchers" have a bias towards confirming what the bosses want to hear - I was just reading this Robin Hanson article about how people don't evaluate the quality of predictions after the fact: http://www.cato-unbound.org/2011/07/13/robin-hanson/who-cares-about-forecast-accuracy/ - but still, I think it would be better to at least consider that there are organisations whose job it is to find the general public reaction to a "brand".

You could find someone who works for such an organisation and suborn them to add an extra question to a proper survey. That way you could gather the reactions of 1000 or 10,000 demographically-representative people in a single action. Let's not waste our time dicking around uploading meaningless data to Google Docs.

A good target in the UK would be YouGov.

I also think it's pointless to worry about a concise definition of rationality until it's been determined that "rationality" is in fact a good brand for public consumption. What if it turns out that the term "rationality" makes 60% of people instantly hostile? Do the research first, then start proselytising.

I find it interesting that the response to this article hasn't overwhelmingly been about criticising Raemon's methodology. Is that because LessWrong members fallaciously assume that attempting to measure the public's subjective, irrational responses to a word doesn't need to be carried out in an objective, rational manner? Or is it, as I increasingly suspect as I edit and re-edit this comment, that I'm a total dick?

Is it just me, or is the word "rationality" highlighted with a yellow background for this post, and only this post? I'm finding it a very distracting effect, and I'm not seeing it anyplace but this one post.

(Firefox 5.0 and Windows XP, in case it's relevant. I'm assuming it's post formatting, but no one else seems to have commented on it, and it seems sufficiently odd to merit mention)

But can you think fluidly in a language you can't speak fluidly? It doesn't follow from being able to understand more than you can articulate that you speak only a small fraction of the words you can think, as byrnema implies. It sounds more like the process of articulation and understanding are decoupled.

Here's two more samples for you. Both subjects are Indian, male, late twenties, highly educated and fluent in English. These are chat transcripts (used with permission):

Roshni: if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
X:Logical ?
Roshni: anything else?
X: Analytical ?
Roshni: would that be positive or negative?
X: Grey area... positive in some case - but in extreme cases, it can be negative
Roshni: if some one person identified that way, which part of the spectrum would you place him on?
X: I need more info
Roshni: let's say there's a club
called the Rationality club
and a group of people attend it twice in a month to discuss various things, do various club activities together etc
X:I'll term them crazy
But I consider most of these stuff as crazy
So I can be wrong
Roshni: there's no right or wrong answer, i'm looking for general impressions only
so negative
X:Yes
..................................................

Roshni:if someone were to describe themselves as a Rationalist to you, what stereotypes would come to your mind about that person?
V:ayn rand first
Roshni:what else?
V: that's about it i suppose. all that would be in my mind is "this is one person i can get along with easily"
Roshni: what if they told you they thought Ayn Rand was a nutcase? well, not a nutcase, just wrong mostly
V: sure
don't matter
Roshni :what if they told you they belong to a Rationality Club
first impressions on the club?
V:........i would really wonder if such a place really adds any value to anyone..........i would consider going to a rationality club albeit with a lot of skepticism
<snip>
Roshni: figuring out if 'rationality' has a negative connotation to people who hear it
V:nah, rationality has no negative connotation to me. i love the idea.
the negative connotation for me is with the word "club"

I think I'll start doing this.

And regarding #4, I think that a phone recording audio in a shirt pocket to aid later transcription wouldn't be amiss. Maybe putting the audio up online would, but in the news/reporting field it's pretty standard practice to have audio going while interviewing.

Also, if you [have/know somone with] a fairly "official-looking" camera with an externally attachable interview-style microphone, then you can go around and do some street spots with less awkwardness. When people see somone talking to a person on the street like that, they get the impression it's one-on-one. And if you approach someone, they'll be more likely to be comfortable with an interview-style conversation.