= 27d567bc2d48d9f40e9ccc20ea370b15)
2014 Less Wrong Census/Survey - Call For Critiques/Questions
It's that time of year again. Actually, a little earlier than that time of year, but I'm pushing it ahead a little to match when Ozy and I expect to have more free time to process the results.
The first draft of the 2014 Less Wrong Census/Survey is complete (see 2013 results here) .
You can see the survey below if you promise not to try to take the survey because it's not done yet and this is just an example!
2014 Less Wrong Census/Survey Draft
I want two things from you.
First, please critique this draft (it's much the same as last year's). Tell me if any questions are unclear, misleading, offensive, confusing, or stupid. Tell me if the survey is so unbearably long that you would never possibly take it. Tell me if anything needs to be rephrased.
Second, I am willing to include any question you want in the Super Extra Bonus Questions section, as long as it is not offensive, super-long-and-involved, or really dumb. Please post any questions you want there. Please be specific - not "Ask something about taxes" but give the exact question you want me to ask as well as all answer choices.
Try not to add more than a few questions per person, unless you're sure yours are really interesting. Please also don't add any questions that aren't very easily sort-able by a computer program like SPSS unless you can commit to sorting the answers yourself.
I will probably post the survey to Main and officially open it for responses sometime early next week.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (269)
No explicit option for asexual and/or aromantic? (Or whatever the term would be for preferring not to be in a relationship, as opposed to not having a preference.)
For estimated probabilities that are greater than 99%, or less than 1%, how many digits of accuracy do you wish the answer to be? Just jump straight from 99% to 100%, or go for 99.999999% if that's honestly the best estimate?
I find this slightly unclear; does 'atheistic Buddhism' count as a 'revealed religion'?
I would be quite surprised if any of us could justifiably estimate any of those questions to an accuracy better than the nearest 1%/epsilon offset.
I regularly think in terms of decibans, where 10 decibans = 90% confidence, 20 = 99%, 30 = 99.9%, 40 = 99.99%, etc. From that perspective, there's as much difference between 99% and 99.9% as there is between 99.9% and 99.99%. Put another way, if 99% is the cut-off, then anything more than 20 decibans of evidence is ignored, even if the answerer has at least 40 decibans.
And yet every year lots and lots of respondents answer such questions with 0 or 100.
Probably not. "Revealed religion" means something like "religion whose teachings are based on revelations from the divine to mankind". If you think that your religion was figured out, discovered, or intuited by a human being rather than delivered to him or her by the gods or spirits, it's not a "revealed religion".
Yes, this means Scientology is not a "revealed religion", because they think Hubbard discovered it. (Scientology is, however, a mystery religion, meaning that it has inner and outer teachings.)
I think the whole point of the world "revealed" is to avoid referring to something like "atheistic Buddhism".
Do you like the cartoon, "My Little Pony: Friendship is Magic"?
(And to compare that with a fandom that seems to be less represented amongst LWers:)
Are you a furry fan? (Reference link: http://en.wikifur.com/wiki/Furry )
It doesn't seem useful to compare MLP to furries without knowing the prior probability that a particular member of the population is a member of either fandom, and to get that prior probability, we need to have reasonably accurate data about the sizes of those communities.
The most prominent social media site for bronies is, arguably, FIMFiction.net; its counterpart for furries is likely FurAffinity.net. (And both sites are similar in that they have a focus on the creation of creative content, while also allowing for journals, commentary, and similar interactions.) To the best of my knowledge, FimFiction currently has about 160,000 user accounts; FurAffinity has around 750,000.
If the results of the two questions turn out to be that there are significantly more than 1/4 the number of bronies as furries, then it's at least arguable that bronies are more heavily concentrated in the LW community than would be expected if there was no correlation between rationality and ponydom. It may not be /strong/ evidence, possibly even only on the order of a couple of decibans, but it seems most likely to be above zero decibans.
The number of furries seems dwarfed by the number of bronies; most estimates seem to suggest at least ten times as many bronies as furries. Compare attendance at the largest furcons and the largest bronycons (it doesn't match the 10x number for a number of reasons, but it suggests the direction).
Okay, we have the largest creative sites being tilted around 4:1 in favor of furries, and the largest cons' latest attendances being 2:1 in favor of bronies; does LW's furry-to-brony ratio fall within that range, or outside it?
So, the ratio might have problems when people fall in both groups; looking at the Brony Herd Census it looks like 17% of respondents were furries, which makes the 10 to 1 estimate calculated other ways seem less likely. (I think most of the estimates I saw for furries predated MLP growing, and it seems likely that the communities grow off each other.)
This question bugged me from last year's survey, but I'm not sure if my objections are valid. Consider a subset of this question: "What is the probability that Bayesian mathematical reasoning was created by a supranatural entity?" It seems... outside the domain, a little bit? Like a question, "What is the probability that Bayesian mathematics is true?"- what would it mean for this to be false? Which is to say, I don't know whether probability estimates are capable of a certain degree of recursive self-analysis, and asking about the origins of Bayesianism may be in that class.
Formidable work. Given recent discussions, might it be worth adding "vegan" to "vegetarian"? (And perhaps even "pescetarian" or "flexitarian" but I suppose one can get lost in small distinctions.)
I'd propose the following three:
The latter, 'reduced meat intake' is intended to represent pescetarian, flexitarian, and meat reduction all in one.
This is a good idea
Alternatively, there could just be the single question 'Are you a vegetarian', and below, a text box next to the question 'if you answered 'yes', why are you a vegetarian?
I don't think a text box is called for, but radio buttons of Ethics/Health/Other might be good.
Not sure whether "aethetics/personal taste" is common enough to be worth breaking out of "Other" but I'm vegetarian because I'm a picky eater and hate the texture of meat.
I feel like using Scandanavia as an example of "socialism" is not really accurate - they're capitalist welfare states with slightly higher taxes than other capitalist welfare states.
Even though the tax rates in America and Scandinavia are similar, the way they're spent is very different: IIUC in the US more money is spent on means-tested transfer payments and in Scandinavia more money is spent on public services like healthcare and education.
Agree. Here is the Heritage foundation ranking of countries by economic freedom. The Heritage foundation's libertarian perspective views Scandanavia reasonably favorably. And they include tax rates in their analysis:
http://www.heritage.org/index/ranking
How would you handle this?
Socialism - workers own the means of production. (Although according to Wikipedia, "There are many varieties of socialism and there is no single definition encapsulating all of them.")
Communism - No markets/the state determines what is produced.
That definition of "socialism" conflicts with the platforms of parties caling themselves "socialist", which is probably what most people using that label have in mind.
There are other confusing things in the survey draft, for example, UK Labour party, which is used as example of liberalism (in the American sense), is affiliated with the Party of European Socialists, the exact same group the Swedish Social Democratic Party (which, I suppose, is used as an example of socialism) also belongs to. Which group do other PES-affiliated parties belong to? Am I correct that by making categories "liberalism" and "socialism", Scott tried to capture the distinction between "mainstream centre-left" and "those that are further to the left than mainstream centre-left" (e.g. GUE/NGL group)? Although I might be unfamiliar with whether left wing people themselves perceive this as an important distinction. I would be very surprised if ~30 percent of LWers self identified as socialists in "workers own the means of production" sense rather than "voting for a PES-affiliated (or equivalent) party" sense.
Just because two parties affiliate doesn't mean they are pursuing identical policies.
If this is intended as a question about existing political clusters, then I think you should merge the "Liberal" and "Socialist" options; I don't think people see them as separate positions (at least in my country) in the same way as Conservative/Libertarian.
If this is really just a question about the left-right economic axis, then let's make it one, viz:
"Which of the following most closely describes your position on economic freedom vs. wealth redistribution?"
Here in France, "liberal" is a slur used by left-wingers against anybody suspected of liking the free market (there used to be right-wing politicians claiming that label, I haven't seen any in the recent elections)
Scandanavia is pretty much what people using the word socialiam in the modern world refer to. Although it have have a different historical definition it would probably cause confusion to use it.
This should presumably be 7 now.
Consider asking people in what year they joined the community? So if you've been here since the start, put 2007.
I would much rather just have to remember a community start date than recalculate this every year. The primary issue is it makes the data a little less clean, especially if you want month-level accuracy.
Extra Credit: Politics, Question about the Great Stagnation: the link is too large Screenshot
Typo in HBD question: "are in fact scientiically justified"
Typo in Great Stagnation question: "See http://en.wikipedia.org/wiki/The_Great_Stagnationfor"
Typos in digit ratio question: "Use a ruler of calipers", "The result is 2D:4D" (should that be 2D/4D)? This question could probably use an image.
Anything I do with gender and sex is going to have lots of people yell at me. But if I keep it the same, it will be the same people as last year and I won't make new enemies.
But to all of us perched on the back of Cthulhu, who is forever swimming left, is it the survey that will seem fixed and unchanging from our moving point of view?
The question about "Country" should clarify whether you are asking about nationality or residence.
Formatting issues:
Question requests:
Other comments:
In case you care about that in order to know which respondents know what they're talking about when answering the MWI question, that's a very poor choice (as I mentioned two years ago IIRC). It basically mostly only checks whether people took QM classes (in many of which interpretational issues are discussed hardly at all) and can remember the tricks to solve second-order differential equations in spherical coordinates. Asking whether people can prove Bell's theorem would be a much better choice. (You should weigh Scott Aaronson's opinion about MWI over mine even though I'm a physicist and he isn't.) Having read How the Hippies Saved Physics, I'd guess that if anything ability to solve the SE for the H atom would anti-correlate with trustworthiness about interpretations of QM when controlling for work status, profession and degree.
Seconded.
Fair enough. In that case, I'll request a question as to whether you can prove Bell's theorem. I guess I was lucky that in my university, interpretational issues were discussed a fair bit in later-year theoretical physics classes.
I think both questions are informative, they just test a different thing.
To give an analogy from copmputer science, the question about hydrogen atom is similar in spirit to, "Would you be able to implement quicksort?", whereas the one about Bell theorem is more like, "Would you be able to reconstruct the halting problem proof?" The latter seems like a much higher bar. I'm curious, do you think there exist many people who can actually reconstruct the proof of Bell's theorem, but who can't solve the Schrodinger equation for the hydrogen atom?
(I'm assuming that by solving the Schrodinger equation for the hydrogen atom, Daniel meant deriving the energy levels of a hydrogen atom from SE, as opposed to say providing the full basis of eigenfunctions including these for E > 0; the latter is much harder and I wouldn't expect most people who took even advanced Quantum Mechanics to be able to do it without looking things up).
I had assumed something in between -- deriving the energy levels, and the eigenfunctions for the levels with E < 0.
Maybe the answers given (either in the Bell theorem case or in the Schrödinger equation case) could be “Yes, right now”, “Not off the top of my head, but I know where to look stuff up” and “Can I prove the what?”
There was a Schrodinger atom question a couple years ago. I'm trying not to keep all questions lest the survey just grow and grow forever. Any particular reason you want to know whether the Schrodinger solving percent has changed since last time?
I mainly just continue to be curious about the correlation between knowing about interpretations of QM and P(MWI). As stated above, I now think that asking about Bell's theorem is a better way of doing that (see army1987's comment and my response to it), which (to my knowledge) hasn't been asked before.
Immigration into one's own country (as for the following two questions) or in general?
Is the "Birth Month" bonus question just to sort people arbitrarily into groups to do statistics, or to find correlations between birth month and other traits? If the latter, since the causal mechanism is almost certainly seasonal weather, the question should ask directly for seasonal weather at birth to avoid South Hemisphere confounders.
I have no idea what the seasonal weather was like when I was born - but I know which hemisphere it happened in.
Not the only possible one. If Alice was born in January and Bob was born in December, she will be 11 months older than him when they start going to school (and their classmates will be in average 5.5 months younger than her and 5.5 months older than him), which I hear can make a difference.
The survey already asks for country. Sure, some people will have been born and grown up in a country in a hemisphere other than that they “most identify with” today, but they'll probably be a small enough minority that they wouldn't screw up the statistics too much.
I think this way of sorting classes by calendar year of birth might also be six months shifted in different hemispheres (or perhaps vary with country in more capricious ways). IIRC, in Argentina my classes had people born from one July to the following June, not from one January to the following December.
In Massachusetts, USA, the classes of people were from one September to the following August, being based on the day classes started.
Some thoughts from my notes when I did the LWSH survey. I mostly plagiarized it from you, so they should be relevant:
Consider adding an option under work and/or profession for "Homemaker." We have at least one.
Consider an option under the relationship goals section for "not looking, but open to the possibility."
There used to be a whole section on akrasia. I note that it is now missing, but it had questions about use of medication and supplements. Those questions didn't specify whether the meds were taken specifically to combat akrasia. They probably should have, given the purpose of the section. If the Akrasia section goes back in, consider modifying it that way.
How about:
Also:
Should I take into account the fact that certain countries make tons of money exporting oil?
(Anyway, it's very hard to answer such a question without doing lots and lots of research, even in you only restrict it to countries with say > 40M inhabitants.)
On "Profession," the field label "Art" is vague. Better would be "Arts and humanities."
"Humanities" includes philosophy, language, and religion; and sometimes history and law, too!
I think what is meant in the survey is specifically the creation of art, e.g. design, sculpture, music, theater, fiction, and so on.
It should be clarified, though.
Yes, it should be clarified. The main ambiguity that I was reacting to is that "art" can mean specifically visual arts or it can mean "the arts," extending to performing and literary arts. As it is, I'm not sure if my profession (scholarship concerning music) is "art" or "other."
In fact (now addressing Yvain again), why is this category called "Profession" instead of "field"? It creates some odd overlap with the previous category of "Work status" which produces a little bit of confusion per my original suggestion and fubarobfusco's reply.
All the time, people put DO NOT CITE on their papers, but that digit ratio web page had a link, explaining why you shouldn't cite: because it's just a copy of wikipedia.
Also, handfacts.com looks really sketchy to me. Not only is it no longer handfacts.com nor e-hand.com, but now eatonhand.com, but it doesn't have instructions on measuring digit ratio.
Why not use the Kinsey sex scale for sexual orientation? (really depends on what you want to gather out of this information)
A split question of whether your children are biologically related to you or just children you are raising.
Allow multiple options (checkboxes) for work-status, and profession.
I have seen "education level" as highest level completed; but it doesn't take into account highest level incomplete. so where I might be part of a bachelors, I could not say so because I have not yet completed it. Perhaps a level of "highest education not yet completed"
An option of "other" for political view
Allow multiple (checkboxes) for referrals.
an option to skip the probabilities section. and submit the survey without participating in that part.
If you are currently working towards a bachelor's, you'll have answered “Student” on the Working Status question.
That makes the data harder to break down; but also someone could be a student + working.
As I suggested checkboxes for that question.
Also what if someone got partway through a bachelors then got a better offer and is now working. the highest level of education /completed/ would not reflect their knowledge/skill.
Good idea. There are even more detailed scales, such as the Klein grid.
"None of your family / friends / acquaintances will suffer from any form of cancer anymore, nor will any of their descendants! How many years of your remaining lifespan would you be willing to give up in exchange for the whole of humanity enjoying the same protection, given your sacrifice would remain anonymous? (If you provide a large number, no life extension would be entailed.)" -- (Requires numerical answer: number of years).
A question on a similar vein I once thought up but forgot to ask on LW:
Omega offers to give you $X, but he will kill you one year from now. What's the minimum value of X for which you'd accept? What if it's one month? Ten years? Thirty years? (No, he isn't going to tell you anything about whether cryonics will work. Also, he promises not to tell anyone about your deal (when you die it'll look like you had a heart attack or something) unless you try to buy life insurance.)
I oppose the Birth month question. It costs a lot of anonymity but provides no good return.
See for example http://www.livescience.com/13958-birth-month-health-effects.html
Bring back the Unreasonably Long Questions section (the one with the personality tests etc.), and maybe add the BSRI to it.
Maybe bring back mother's age and father's age in the Super Bonus Questions.
What about a question about PUAs in Extra Credit: Politics?
How would you word that question?
By analogy to the other questions, it should be something like "How would you describe your opinion of pick-up artistry, as you understand the term?" and either a link to an external page describing PUA or a short inline explanation as with the HBD question. You could try to tease apart descriptive and normative claims, but note that no such distinction is attempted with e.g. the feminism question.
I don't think that's good. PUA isn't simply a political view. It's a set of behaviors and if you want to know whether someone engages in them, the label isn't the core issue.
P(Aliens) is P(Aliens) and not P(Aliens|Simulation=False)
That's not a terribly rare sentiment among non-poly single people either.
It seems to me that unless a new person shows up and you say "sorry, not looking for more right now" you're in the first category. But separating out "looking and receptive," "not looking but receptive," and "neither looking nor receptive" seems potentially useful.
It would also be very interesting to see whether those who think that MIRI is more effective actually donate more.
At best I think we should directly the corresponding test questions into the census. Is there a particular set of questions that would be ideal for this purpose? I also oppened a question on cogsci.stackexchange for that purpose.
Any particular implementation details on OCEAN? Exact same as last time?
I'm not sure. Didn't we have some odd results last time? It may be better to use a different website to see if the aggregate results or the results from retakers differ with last time.
I suggest including the Big Five Inventory (BFI) in the survey itself. I've created an example of this on Google Forms. (I've reordered the inventory such that the first 11 items constitute the BFI-10, so that respondents can choose between the 44-item and 11-item versions).
The BFI is the inventory that was used in the online test to which the 2012 LW census linked. See also my comment about this in the 2012 LW census thread.
Why not directly include the 10-item Big Five in the survey itself?
I strongly agree with the basilisk suggestion; have you provided Yvain with a specific question and set of answers to use, per his request, or can you do so?
Or would you not be satisfied with including it as a Super Extra Bonus Question, wanting it to be in the main body of the survey?
I believe I did so, but Yvain is a smart enough cookie that I don't really need to draw up a list of exact phrasings. The question is not how exactly the question will be written, the question is whether such a question will be allowed at all.
If it is to serve its intended purpose, it would be much better to get it in the main body of the survey to defang the sample size objection.
He said “Please be specific - not "Ask something about taxes" but give the exact question you want me to ask as well as all answer choices.”
The basilisk question is an exception.
No it isn't
Agree with this. Big 5 is worth having.
I would personally be interested in more detailed drug use questions. Examplae. In the Past year have you taken:
-Modafinil
-Amphetamine (of any sort including Adderall)
-Heroine/Other Opiates (not prescribed by a doctor as a painkiller) -Marijuanna
-Hallucinogens (LSD/DMT/Psychobillin)
-Testosterone/HGH (if you took testosterone because you are transexual do not click yes)
Maybe there is a better list of drugs?
Alcohol too.
Let's see ...
Idk how long of a question Scott Wants this to be. If Scott is ok with a very long list than I too would prefer that. You list looks reasonably good.
Though I would include testosterone. People take this for reasons other than muscle growth.
Admitting to taking medication for treatment of a mental condition can be very bad for your career. If these questions are asked, the results should be segregated from other columns to prevent deanonymization.
Admitting to taking illegal drugs can be, too. Some forms of prejudice are even more institutionalized than others.
I had this last time, and several people told me to take it off because it was bad to make people admit to illegal activities.
Also, for complicated reasons I can't do "Check as many as apply" questions, so this would take forever.
Well, nicotine, caffeine and alcohol are legal in jurisdictions most LWers come from, so you could at least ask about those.
You could use randomized response method for this question or any other controversial question (for the type of questions that are technically feasible, of course).
Wouldn't that make it pretty much useless for anything other than estimating the number of people who would answer in a certain way?
:(((
You have the "Slate Star Codex" question under the "privacy" section. You may want to retitle the section to something like "top matter".
I'd like an "apolitical" choice for my political views. Especially here, there's likely a significant slice of the population who choose to not have any politics in the manner of Paul Graham, and thus equally don't identify at all, not even a little bit, with any of the given political options.
"If you were referred here by another blog—" "another" is needless.
Consider asking a "% positive" followup to total karma.
I'm going to ask for 4 options in the mental health section; there should be a separate category for people who weren't diagnosed with depression but believe they have had it.
I'd like more resolution in the extra credit section on politics. Just moving to a 7-scale would be a large improvement.
If the survey occurs near/during a HPMOR update (particularly a large update), your responses to the HPMOR questions will be skewed by people who would've read all of it, and will have read all of it, but aren't quite there yet because it just released.
The semicolon in "Hours of TV" should be a comma (sorry for being all nitpicky, that one just really bugs me.)
Instead of an specific apolitical choice, simply "others" might be useful.
I think that "None" and "Other" should probably be kept separate options.
Possible drawback: this will make it easier to work out who any given response is from, which some people might be uneasy about.
Please change the "referred by a link on another blog" option to "referred by a link on another blog or website". It's been bugging me for years.
Again, your relationship style question conflates very different clusters. There is a big difference between traditional monogamy and serial monogamy, or between the ideal of polyamory preached in rationalist circles and the harem type of polyamory (polygyny, mistresses, etc...) which is traditional for a sufficiently high-status man to have. The obvious way to solve this is to split the "prefer monogamous" and "prefer polyamorous" answers into two answers each, each of which describes the relevant clusters in a short sentence similar to the answer choices in the political question.
You are asking too many calibration questions for me. I might answer 1, 2, or, at a stretch, 3. 10 questions is gonna cause me to leave that whole section blank.
Likewise, I don't think I'm gonna answer the probabilities section unless you switch to radio buttons, like you use in the extra credit political section.
Per your own advice, you may want to add a completely implausible question to the survey so you can measure our Lizardman's constant and subtract it from the other results.
I'd suggest a section asking opinions on famous thought experiments. "Do you flip the switch in the trolley problem?", "do you push the fat man in the modified Trolley problem?", "is the China brain conscious?", "do you one box against Omega?", "does the Star Trek transporter kill you?", etc... maybe some not so famous though experiments, too. Eliezer has that thing in which a primitive culture which is afraid that cameras will steal your soul but does not realize that the eye is basically a camera is used as an analogy for crbcyr jub ner nsenvq bs hcybnqvat, and you asked that question about the angel which offers you reincarnation as a metaphor for pelbcerfreingvba.
My bonus question is "Do you read non-rationalist fanfiction? That is to say, do you read fanfiction other than HPMOR, Luminosity, Friendship is Optimal, etc... and their derivative fanfanfiction?"
+1
I would like that the question for previous surveys asks for the private key that was used in the last survey if the person remembers.
Question about politics is lacking a "none of the above" option.
That's how it must be. The question is about which you most identify with. "None of the above" would be confusing in the context of that question.
I disagree even a question of what you "most identify with" carries a connotation of a non trivial identification with what you would answer. At least it does to me.
We want to avoid a situation in which 90% of the people choose "none of the above."
Some questions I'd personally put in if I was writing this survey (you don't need to put these in if you don't want to, just suggestions):
How would you rate your general success in life in the following areas, before and after you became a member of this community?
I'd add: "Do you use/have you used a Spaced Repetition program, like Anki or Supermemo?"
In the politics I'd add a "how strongly do you care about this political affiliation" (scaled on 1-5, from "not at all" to "it's an important part of my identity")
A slightly longer option is to ask people to rate every political affiliation from 1 to 5. This lets us identify clusters better, as well as total political identification (if someone gives 1s to everything, that's the 'apolitical' option).
I like this idea. If implemented, it might be wise to retain the old-style political question (maybe explicitly labelling it as such) as well, for backward compatibility with past surveys.
I'll write down a long list of questions. I think they are useful. If you don't like them all because they are too much I would certainly like to keep data about spaced repetition software usage. The question from last year was good.
I would specifically like to have the bolted questions included. With them we could cut the question that asks for self identification as vegetarian.
Given the absence of perfect information our justice system has to convict a few innocent people. Where the optimum of the ratio between guilty going free and innocent getting sentenced? X guilty going free for one innocent being arrested. What's your X?
Morality Questions:
How morally problematic are on a scale of 1 to 5 the following actions:
1) Eating animal products
2) Discriminating against a person based on their gender
3) Abortion at 3 month of age
4) Abortion at 7 month of age
5) Not voting in a national election
6) Rerouting a trolley car to kill one person instead of five.
Options: today, yesterday, 2 to three days ago, up to a week ago, up to a month ago, up to three month ago, up to a year ago, up to three years ago, up to ten years ago, more than ten years ago, never +++
When was the last time you read a scientific paper beyond the abstract that was work related?
When was the last time you read the abstract of a scientific paper that was work related?
When was the last time you read the abstract of a scientific paper that wasn't work related?
When was the last time you read a scientific paper that wasn't work related?
When was the last time you read a science textbook?
When was the last time you used spaced repetition software (Anki, Memosyne, Supermemo etc.)?
When was the last time you made an effort to record personal data for future analysis?
When was the last time you took a Vitamin D supplement?
When was the last time you took a multivitamin supplement?
When was the last time you took a Iron supplement?
When was the last time you took a Creatine supplement?
When was the last time you took a Adrafinil/Modafinil supplement?
When was the last time you drunk alcohol?
When was the last time you smoked?
When was the last time you meditated?
When was the last time you prayed?
When was the last time you got the flu vaccine?
When was the last time you ate flesh?
When was the last time you went the gym?
When was the last time you went jogging?
When was the last time you went dancing?
When was the last time you exercised in any form?
When was the last time you drove a bicycle?
When was the last time you drove a car?
When was the last time you used public transportation?
When was the last time you donated to charity?
How often do you exercise on average per week? {Enter number}
How often do you eat on average meat per week? {Enter number}
(this question allows together with the "When was the last time you ate flesh?" question allows us to to see to what extend people lie to themselves - are sub groups who score higher on the calibration questions better able to estimate their average meat consumption?)
I like the moral rating questions, but I think they should have a scale that includes both positive and negative values. Asking "how good or bad is X?" will get more honest answers than "how good is X?" or "how bad is X?".
You are probably right.
The core things I care about isn't direct honesty. I want to see how the difference in judging eating animals translates into rating of how much animals the person eats. That means I need a big scale where a lot of people can answer that eating animals isn't ideal but not that big of a deal. I also don't know that anyone argues that humans have a moral obligation to eat animals.
On the other hand in the case of rerouting trolley cars a two sided scale seems important.
That depends on which crime we're talking about -- somewhere around 10^6 for illicit drug use, 10^3 for shoplifting, and 10^0 for murder.
Does Duolingo count?
Given that you also ask about multivitamins, I guess you mean ‘a vitamin D supplement not as part of a multivitamin’, but you might want to say it explicitly.
Do e-cigs count?
Language nitpicks: “eat meat [not flesh]”, “ride [not drive] a bike”.
I would prefer not counting it.
Whether it's bundled isn't that important. On the other hand what's interesting would be the dose. But that add another question.
"flesh" feels like a gratuitously loaded term. Is there any reason for that question not to simply be "when was the last time you ate meat?"
I have no problem with changing it to meat.
FYI, when I read your abortion questions I was unsure whether you were counting months post conception or post birth. Timing of pregnancy is more often discussed as trimesters or weeks.
I don't think abortion post birth is a thing. If it's easier to understand I have no issue with changing it to 12 weeks and 28 weeks.
"After-birth abortion" turns up sometimes in thought experiments and philosophy papers, and my prior on weird thought experiments turning up on lw is pretty high.
Actually, I remember hearing someone mention that in general LWers are more okay with Infanticide (or post-birth abortion if you prefer) than the average population. The reasoning, I assume, being that their self-awareness is more similar to animals than to a human adult and that you aren't really destroying a full human consciousness. I don't remember where it was posted, but i do remember it sounding like someone was summarizing LW survey results.
Basically, what I'm saying is that I wasn't sure either given the phrasing, since I've heard this kind of thing discussed before on LW.
Yeah, the infanticide thing is a classic Peter Singer bit which I imagine a lot of people on Less Wrong have heard and considered before. I think the standard counterargument is that we need a good Schelling fence for when it's okay to kill people - pro-life folks would argue that it should be at conception or something, pro-choice folks would say it should be at birth, but so far nobody's come up with any reasonable one that comes after that. So infanticide should be disallowed for societal reasons, even if we allow that it might be acceptable in various hypothetical scenarios.
It might be one of those Europe vs US (or maybe continental Europe vs Anglosphere) things -- pregnancies are usually measured in months here in Italy too.
A question on romantic orientation would be good.
What do you mean with "romantic orientation" as opposed to sexual orientation? I don't think the term is well known and we already have enough questions about that domain
What gender/s you are romantically attracted to, and also how strongly you feel that attraction, see the Wikipedia page. It is mainly useful for asexuals (and also, I imagine, people who answer 'other'), but it's certainly possible to have a romantic orientation that doesn't match your sexual orientation. Maybe it could be included as an optional write-in box, or at the end?
Could you provide a source for that claim that there is a sufficient number of people for which that distinction is useful?
In the last survey, there were 47 asexuals and 39 'other's. It is a useful distinction for asexuals and I imagine it would be useful for many 'other's. Furthermore, as per philh's reply to RichardKennaway's comment, the distinction is probably useful for some non-asexual people.
Whether this is a sufficient number of people to add an extra question is a bit of a more thorny question. For comparison, there were 25 trans people in the last survey, fewer than the number of asexuals, and there are options for them in the gender question. Even if it's too onerous to add to the main sex/gender/relationship section is too onerous, I think that it could find a happy home in an extra credit section.
Okay, 86 people seems like enough to be able to separate them into smaller chunks.
I'm unclear on the distinction between "sexual orientation" and "romantic orientation". I can understand the distinction between "sex" and "romance", but the two are strongly connected to each other. Are there people who want sex with one gender, but romance (whatever that means without sex) with the other? The Wikipedia article, um, isn't helping me.
My impression is that a person's romantic orientation is almost always a sub- or superset of their sexual orientation. At any rate I don't recall hearing of anyone who identified otherwise. But the inclusion can go either way (e.g. asexual but homororomantic, or bisexual but heteroromantic). They're strongly correlated but distinct.
There are also non-asexual people who are a- or demi-romantic.
The "currently looking for more relationship partners" feels to me ambiguous -- what if someone is open to having more relationship partners, but not actually looking for more? Perhaps make this a three option issue "actively looking" vs "open to having more relationship partners but not actively looking for more" vs "not open to having more relationship partners"?
For relationship status, a polyamorous person can be married and in a relationship at the same time, which is a problem. Similarly, someone can be living with their partner/spouse and additional roommates. Also, "Liberal" in the Political section should probably be renamed to "Progressive", to avoid collisions with how "liberal" is used in Europe and in political philosophy.
In the mental health category, I'd love to see (adult) ADHD there as well. I'm less directly interested in substance abuse disorder and learning disabilities (in the US sense) / non-autism developmental disabilities, but those would be interesting additions too.
For Super Extra Bonus Questions: (feel free to modify the answer choices)
With which of these metaethical positions do you most identify?
With which ethical position do you most closely identify?
With which of these broad political groupings do you most closely identify?
Could you have less fine-grained answers, so that I don't have to spend a week on the SEP just to know what the answers mean?
If you want less fine-grained answers, there's the consequentialism/deontology/virtue ethics question in the earlier part of the survey.
"Moral nihilism is the meta-ethical view that nothing is intrinsically moral or immoral." (http://en.wikipedia.org/wiki/Moral_nihilism) Utility functions (aka morality) are (is) in the mind, not in Nature. That would probably be the answer of most LWers. Otherwise, you'll have to tell me what you mean by morality.
Is utilitarianism used as "maximizing happiness" or "maximizing utility". If it's "maximizing utility", well isn't that everyone's position? What differs is simply what counts as "utility".
That is an inaccurate definition of nihilism because it doesn't match what nihilists actually believe. Not only do they reject intrinsic morality, they reject all forms of morality altogether. Someone who believes in any kind of moral normativity (e.g. a utilitarian) cannot be a nihilist.
Utilitarianism is used as "the normative ethical theory that one ought to maximize the utility of the world". This is in contrast to something like egoism ("the normative ethical theory that one ought to maximize one's own utility") and other forms of consequentialism.
Thank you for your answer.
Do nihilists think they have no goals (aka terminal values) or do nihilists think they don't have goals about fulfilling others' goals or is it something else?
Ok so would that be right to say this?: Utilitarianism is giving equal weight to everyone's utility function (including yours) in your "meta" utility function. Egoism means you don't consider others' utility function in your utility function.
And then there is everything in-between (meaning giving more weight to your utility function than to other's utility function in your "meta" utility function).
I am not a nihilist, and I don't know if I'd be able to pass an Ideological Turing Test as one, but to give my best answer to this, the nihilist would say that there are no moral oughts. How they connect this to terminal goals varies depending on the nihilist.
The first part, kind of, the second part, no. The utilitarian holds that the right thing to do is determined by what maximizes world utility, which is produced by utility functions. All utility, including your own, is given equal weight in the "moral decision" function. As for egoism, it simply means that you consider others' utility functions to the degree that they're a part of your utility function. It doesn't mean that you disregard them altogether.
Or “Let's please not murder”, which doesn't express a proposition either.
This implies that you are only poly if you're in more than 1 relationship. I suggest simply
because
Someone? Who is someone? Honestly, I'm curious, because I can't think of who these someones would be. What research psychologists who aren't already members of Less Wrong pay attention to it? I doubt it would be Scott using it to publish something(?) Maybe he shares the results on Slate Star Codex, and a psychiatrist friend of his considers Less Wrong an interesting subject pool, so they use the data? This is something I'd want to know before I give consent. Like, even a rough description of who Scott conceives of as using this data would make me more comfortable.
For the 'Gender', and 'Sexual Orientation' categories, why not allow the 'Other' radio button to instead be a text box subjects can fill in themselves?
I've got a hunch that within the Less Wrong community there's enough of a diversity of mathematicians that we'd discover something interesting if the type of career as mathematician as subject could choose from was split into two options of 'applied mathematics', and 'pure mathematics'. By interesting, I mean the ratio of one type to the other might reflect how the community thinks about mathematics, especially regarding issues relevant to the MIRI. From there, we could try to infer what's going on. If nothing interesting is going on there, then my hypothesis is falsified, and the survey tried something new that in no way threatened its integrity.
The Political Compass Test generates quantified results on a two-dimensional axis:
x-axis: left-right y-axis: authoritarian-libertarian
This is my graph as a sample. Taking the Political Compass Test can be done in five minutes. If greater quantification allows for much more potential value to glean results by mining the raw data, then an extra five minutes on an already long survey is a small price for subjects to pay. There could be two text boxes for each subject taking the survey to input their scores along each axis into them, and then, if there's a spreadsheet program behind the survey somewhere, it can organize the results for you. What Scott can tell us by having that data analyzed however it can might be more useful than the discrete radio buttons of surveys past.
If this sort of thing is worth doing with made-up statistics, imagine the value of information we'll get by using real ones.
If that's too much data for Scott to handle alone, then we can ask our friend Peter Hurford, the data scientist with a double major in psychology and political science if the effective altruist distributed volunteer task force he coordinates, .impact, would be willing to help. Honestly, I can only imagine Scott is incredibly busy, and outsourcing it to a trustworthy few willing to do so with their own spare time anyway might be worthwhile.
I personally would appreciate if the 'Moral Views' question could be converted to a rating system. There could be set of Likert scales for how much you identify with virtue ethics, consequentialism, and deontology. Or, instead, there could be a ranking system for which you identify which ones you feel are closest to your own, from most to least. Again, this offers a finer grain of data, so more value of information to be had.
Part Five keeps me out because I've never taken an IQ test, and getting one off the Internet can be too expensive. Additionally, I'm Canadian, so the scores on all the other tests aren't relevant to me. Not that I mind, much, but it might be something to think about for next time, as maybe at least 1/3 of subjects won't be able to respond that way.
my opinion as a single data point: this survey is just long enough. I've learned a lot more on Less Wrong since last year. My technical comfort zone has expanded, so when I read questions about, e.g., assigning probabilities to future events, my eyes don't glaze over as much as. So, the survey feels shorter than last year. I remember the survey from last year was so long that I didn't even bother.
If you're including a poll on opinions of feminism, I'd be interested to see one on the men's rights activism movement as well.
Suggestions for Questions Specific to the Center for Applied Rationality
Maybe take the opportunity to ask more questions about CFAR, and ask the CFAR staff you know if they want questions answered, because they love data. As a CFAR workshop alumnus, I'm interested in seeing where that community is headed. Here are some suggestions:
If you're not one of the CFAR alumni, what's your estimate that you will attend one of their workshops in the next [six months, one year, two year, insert timescale here]?
If you haven't attended a CFAR workshop, do you ever intend to?
If you don't want to attend a CFAR workshop, what's stopping you the most from going?
If you're unsure if you would want to attend a CFAR workshop, what would increase your confidence that it's worthwhile to attend one?
[Likert scale, 1 to 7]: 1 being 'not very different', and 7 being 'very different', what's your opinion of how different is rationality as it is expressed on Less Wrong from applied rationality as it is taught by the CFAR?
I've been reading the thread, and it appears there are some categories of questions Less Wrong wants explored in more detail, with more questions. These subjects include:
This is all very interesting. However, with upvotes, below all these are comments of the form "I'm not taking this (part of) the survey if you add a bunch more questions, each with more options to consider than I even understand what they mean."
May I suggest that we use this survey as a general value of information survey, and, if we really want to later, based upon the results of this initial one, more specific ones are made?
For example, if we want to discover exactly which one of flexitarian, vegan, pescetarian, or plain vegetarian someone is, we can ask that if, like, >5%, or 10%, of the community reduces meat intake.
or
there could be five checkboxes for drugs, such as
Then, on the next survey we can ask more details about specific nootropics if it turns out half of us use them everyday, and that's exciting because we didn't see it coming, or whatever.
or
Ask a question about whether someone is grey aromantic demisexual, or whatever, if the first survey returns a result of more than literally only 3 people choosing asexual on the initial survey, because frankly asexual people are rare, as much as they are special, lovely people like everyone else.
This second survey could be called, I don't know: Less Wrong 2014 Census Part 2: Zooming In on Bonus Sections, and it could be taken by the few dozen people who differed widely from the mean in the first survey in the various sections. Of course, if Scott doesn't want to make that survey, he shouldn't have to. We can do it, or at least somebody can, if they really want to.
Additionally, I think there could be a Less Wrong Philosophy Survey, or Less Wrong Prediction/Calibration Survey, simply because some among us could make awesome surveys just as grand as the original Less Wrong census in its own right, and that there may be sufficient demand for that from the community.
Somebody upvoted my above comment. At least one person liked this comment. I grouped all my general suggestions for changes in one comment because I thought I shouldn't clutter the thread with all my ideas in their own separate comments. I figure people skimmed over the long comment to upvote terser, better suggestions, which makes sense.
Whether you upvoted the above, or not, if an single example is an excelletn suggestion, build upon it in your own new comment, or let me know. Maybe a put up a bunch of noise above, and it's difficult for us to send the signals. I just want to send my best signals through as suggestions to the survey like everyone else.
The "with family" option in the "living with" question is ambiguous for those of us with children. I suggest changing it to "with parents or guardians[1]", changing the partner/spouse option to "with partner/spouse (and children if applicable)", and adding an "other" option for less traditional living arrangements.
Questions in the mental health section are inconsistent about whether they're referring to whether you have ever suffered from a condition ("have you ever been diagnosed...") or whether you are currently suffering from it ("...I personally believe I have it"). Some are lifelong conditions, but others like depression are temporary.
Questions on feminism / social justice / human biodiversity don't distinguish between what you think of the concept itself and what you think of the movement around it. (Or is this the point?)
[1] Is this a Britishism? Feel free to change it to the equivalent in US English.
I live my with my children but not with a partner or spouse, so I'd want to see even more family arrangements, since I don't think single parenthood is unusual enough to be lumped in with "other."
I'd propose “with parent(s) and/or sibling(s)”, “with partner(s)/spouse(s) and/or child(ren)”, “with roommate(s)”, “alone” and “other/I'd rather not say”. Maybe also “in communal sleeping quarters, e.g. students' dorm or nursing home”.
May want to add Slate Star Codex as an exception to the referrals question.
Time in community question needs to be updated to 7 years for the start of the community.
Might be worth it to specify aggregate Karma if you have multiple accounts. (This is an account that I started using after I decided I no longer wanted to use my real name. I mostly lurk anyway, though.)
It would be worth it to add a "no meetups in my area" option to the meetups question.
The header for part eight is listed twice.
Please make the options:
Before the professional intelligence measurement question, please add something like:
It might make sense to add on something like:
Or, alternatively, we could get at the same sort of question by asking this:
But it's not obvious to me that this is a thing people would be good at estimating.
I like the group-of-adults formulation, it's clever and I think it's more amenable to getting at people's real estimates.
I expect people to have a better idea of group sizes than normal distributions, but the worry is with 'chosen uniformly at random,' since people tend to be in IQ bubbles. If you can think of 100 people you know dumber than you, and 10 people you know smarter than you, then you might say "well, that implies a group size of about 10," without realizing that half of those people you know from graduate school where the average IQ is in the top percentile.
But it seems like we can compare that to SAT scores and measured IQ (if provided) and that'll give us useful information.
I've also edited it so it's a little less ambiguous (were you randomly selected before or not?) but I want to add the super formal mathematical version to make it totally unambiguous, which may or may not be a good idea.
Yeah, I'd expect some pretty serious regression to the mean, or more accurately to slightly above the mean, going on with the group-of-adults method. People of low intelligence hang out with other people of low intelligence and probably have a slightly inflated opinion of their relative capabilities; likewise for people of high intelligence. But people outside your social circle are totally invisible.
At least, assuming people actually use the group-of-adults method, which I don't really expect them to.
That is, you would expect people to say "well, I want my IQ to be 145, so let's calculate the percentile for that, and then use that percentile to calculate the group size"? Or that they'd just leave the question blank?
I'd expect a lot of people to leave the question blank if there are other IQ questions. I'd also expect a lot of people to work backwards from scores, more or less formally -- I don't think I'd expect many people to actually do the math or track down a normal distribution calculator, but thinking like "well, I got 99th percentile on that standardized test in high school, so I'll say 100" would probably be common.
To be fair, that's probably more accurate than what you'd get by counting up the number of people you know who're smarter than you.
Meh. If I was asked that question, I'd try to guesstimate how many people smarter than me there are in my town and divide by its population.
That depends a lot on the group of adults you use as reference. World population? The population of the country in which you are living? City? Your facebook friends?
I think IQ is much better because there are objective standards. "Smart" is also a word that different people interpret slightly differently. IQ is much more precise.
Ideally, ethnic Brits, since I believe that's what's typically done for IQ reference distributions.
Yes, but people may be better at imagining groups than normal distributions. Just how much unlikelier is 150 than 145?
That would also be interesting to test. Ask both for the IQ and ask for the relative intelligence.
Do well calibrated people perform better on this task?
A couple (more) questions I'd find interesting:
How many times do you exercise per week, on average?
How many nonfiction books do you read per month, on average?
How knowledgeable would you consider yourself in the following fields? (on a scale of 0 to 5, where 3 is about "studied it at university" and 5 "I'm a publically recognized expert")
Psychology
Economics
Artificial Intelligence
Statistics
For the first one, it might be better to ask how many hours / minutes rather than how many times. Otherwise somebody's 10-minute cycle to work is counted with as much weight as somebody else's 2 hours in the gym.
(agreed, and the question should probably be formulated to distinguish deliberate exercise from things like cycling to work)
I prefer asking for the times of exercise per week and a question for when the person last exercised to check whether certain people overestimate the amount they exercise. It would be interesting to see whether people who are better calibrated on the calibration question do better at this task then people who aren't calibrated.
I also don't think that minutes are very meaningful. Intensity matters as well. It's easier to stay with the simple question for the amount of exercise. I don't think there a strong systematic bias at play.
I like the calibration check idea, and it's a fair point about intensity. The last survey I took that included this kind of question asked about "moderate exercise (eg brisk walking)" and "intense exercise", or some similar wording, which I thought was a reasonable split. These might all be details we don't care about though.
Good questions. Especially about excercize. Studies allways seem to show big correlations with that so its worth asking about.
Maybe you could split “Yes” in the Blood question into “Yes, in the last 12 months” and “Yes, longer ago”.
I'd been interested in seeing how many people participate in systematic self-improvement efforts outside the LW/CFAR space. Some examples in rough increasing order of weirdness: diet, fitness, psychiastric help, life coaching etc., social skills programs of various kinds, nootropics. This covers a lot of ground and there's probably some options I haven't thought of, but these two questions would be a good start:
(this would probably need "I tried and it didn't work" and "I tried but found it too hard" options. We'd probably also want some way of distinguishing between "I'm allergic to onions" and "I kind of try not to eat red meat" and "I'm full-blown hardcore paleo".)
I shall repeat my request for a second question under "Degree" that asks about one's highest degree attempted or in progress.
It seems to me that the distinction between "started work on degree X, but abandoned it" and "currently working towards degree X" is almost as large as that between either of those and "never attempted degree X". (Likewise for qualifications that aren't degrees.)
So if we really want more resolution than a single question will get us, it seems like we need three questions to capture all the information. If so, maybe they should be (1) highest degree attained, (2) highest degree attempted or in progress, (3) estimated probability of completing #2 if different from #1. So if you've abandoned a degree, you'd put 0 or something close to it for #3; if you're still working on it and fully expect to complete it, you'd put something close to 100% for #3; if you're struggling through a PhD and uncertain of the future, it might be more like 50%. This gets (aside from some unusual cases where someone has both abandoned a degree and started on a new one that they haven't finished yet) strictly more information than "highest attained; highest attempted; highest in progress" -- but at some cost in comprehensibility.
If having three questions is excessive but having two isn't, perhaps the following would be a good balance: (1) highest degree earned or in progress and confidently expected to be completed, (2) highest degree abandoned or in progress but quite likely not to be completed.
If only one question, I think asking for highest degree actually earned is the best option.
The standard on psych studies seems to be "Level of education: (1) Some high school (2) High school graduate (3) Some college (4) College graduate (5) Some post-graduate (6) Graduate degree." This is pretty simple and does not warrant more than one question.
I personally have no problem with that -- but komponisto wants to make more detailed distinctions, and was originally (i.e., at the other end of the link in the great-grandparent of this comment) responding to someone else who wanted to count courses currently in progress as well as ones already completed.
I'm sure both of them have reasons (indeed, it's not hard to guess some) and I bet they're both aware that it's usual simply to ask for highest qualification actually attained.
First of all, the survey is excellent as it is. So I, too, merely suggest an addition.
I'd like to see added something about teaching / spreading rationality / proselytizing. My first idea would be something like: "Do you attempt to make others think more rationally?" with options on a spectrum from "no" and "I generally try to be the voice of reason in discussions" to something like "Many know me as an advocate for rationality". Not sure about how to adress an online/offline divide.
Thanks so much for running this again
Comments
-Items should either be randomized or if sorted, they should be sorted based on last years prevelency. For example in politics libertarian is the top listed item despite not being alphabetically first or being a majorty / plurality view.
-The religious denomination shouldn't ask atheists to skip the question. One is about what you believe the other about what you do. Plenty of atheists fast on Yom Kippur or go to church on Christmas.
-Less wrong use should have two items on comments that differentiate by frequency. Eg commented on a thread and comment at least monthly.
-Instructions for percentages should be more clear. Maybe give an example. As worded I'm unsure if to write"50%" or just "50"
-Not sure if cryonics should be conditional on no catastropic event.
-While we are asking about all the psych issues any reason not to ask about anti social or ADHD or other ones as well.
-The taxes question isn't meaningful. Politicaly the divide isn't lower higher in general. It's typically more / less progressive. Suggestion; "Should taxes on people earning more than 1mm per year be lower / higher"
-Older and younger siblings should be next to each other and not separated by birth month
-More details on vegetarianism. Rather than yes / no it should be a spectrum. Maybe: "vegan / vegetarian / reduced meat (eg meatless Monday) / no vegitarian leaning"
That should be Yom Kippur.
Oops. Thanks for pointing that out. I'll correct it.
Not exactly a question per se, but I remember Yvain complaining about invalid entries, such as words in the number field etc. etc.
This sounds like something that simple Regular Expressions could speed up. Perhaps we could ask him what his current survey workflow is and try to see where they can fit?
Although this advice seems a bit TOO obvious not to have been mentioned before. Apologies if so.
If there's no good way to ask about specific supplements in the survey, would anyone mind if I raised it as a question in Discussion?
How about a survey question about how much people care about their LW karma?
Don't you want devoted followers?
Who leave their families for you
Give their money to you
Give their bodies to you
Give up their lives for you
Consider you God, and will kill for you
Don't you want to become a cult leader?
Since the death of God there has been a vacancy open
You can fill that void, here is how
To the Super Bonus Questions, could you add:
Is there a number which you have considered your "favorite number" for some period of time (i.e., from before you read this question)? If so, what is it?
I would really like to see these questions in the survey:
For the questions:
The questions are:
And maybe also this one:
EDIT 1: I've removed this specification: where death = permanently not conscious; if you create a clone or a simulation that is not a direct upload, it doesn't count as 'still living'.
EDIT 2: I've added that one could control its aging process in 2a).
A random comment.
This is the first time I've seen "anti-agathics". Based on what I know of biblical Greek, I read this as something that would be like "anti-good". If I had been in charge of making up an anti-aging drug, I would have called it something like anti-presbycs (maybe that wasn't chosen because it looks too much like "presbyterian"? Presbyterian does derive from the world meaning "elder"...).
This isn't a request to change the wording if that's what people who will be taking the survey are familiar with BTW, just something I noticed. Carry on.