Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

2017 LessWrong Survey

21 Post author: ingres 13 September 2017 06:26AM

The 2017 LessWrong Survey is here! This year we're interested in community response to the LessWrong 2.0 initiative. I've also gone through and fixed as many bugs as I could find reported on the last survey, and reintroduced items that were missing from the 2016 edition. Furthermore new items have been introduced in multiple sections and some cut in others to make room. You can now export your survey results after finishing by choosing the 'print my results' option on the page displayed after submission. The survey will run from today until the 15th of October.

You can take the survey below, thanks for your time. (It's back in single page format, please allow some seconds for it to load):

Click here to take the survey

Comments (70)

Comment author: berekuk 05 October 2017 01:11:08PM 1 point [-]

So, what happened?

This post is hidden from Main and the survey "is expired and no longer available", even though the post mentions that it should run for 10 more days. I wanted to share it with Russian LW community, will it be back in some form later?

Comment author: ingres 07 October 2017 02:06:34AM 0 points [-]

Right sorry, I got distracted by life a bit there. I'll write up a post explaining what happened to the LW Survey soon and where I'm planning to go from here.

Comment author: MaryCh 17 September 2017 02:42:18PM 0 points [-]

Thank you for compiling another survey!

And on a completely unrelated note, what is it that we actually want to know about ourselves as LW? Surely it can't be the gender ratio. It's not like we don't already know not to post "traditionally feminine" stuff or something. It seems to me that surveys aren't done to achieve some further goal, although the results, of course, are of some curiosity. Sorry if this is counterproductive, I am genuinely interested in the above question.

Comment author: gjm 18 September 2017 12:18:22PM 2 points [-]

I think it's mostly just curiosity.

There are some things that make useful responses to common criticisms of the community. "You think you're so smart, but you really aren't!" (People reporting "proper" IQ scores and SAT results here consistently come out a couple of standard deviations above average.) "LW is a hive of racist sexist neoreactionary scum and villainy!" (LW consistently comes out more liberal and more feminist than the population as a whole. It has a lot of gay people and a lot of trans people. On the other hand, it doesn't have a lot of women or black people and some specific ideas neoreactionaries like are more popular here than in the world at large.) "You're a bunch of borderline autistics!" (Er, um, yeah, that one seems not to be too far from the truth.)

Comment author: Elo 17 September 2017 07:52:15PM 1 point [-]

Without explaining - the demographic is useful to know.

I think it's good to ask the question - if the question yielded x results what would we do with that information?

For gender specifically non-binary gender is about 10x more common on lesswrong than in the American population. That's worth knowing.

Comment author: MaryCh 18 September 2017 07:50:38PM 0 points [-]

But why is it worth knowing?

Comment author: Elo 20 September 2017 09:33:05PM 0 points [-]

For that subset of the demographic there may be use in posts on relevant topics. Just as we have higher (double) depression rates than the normal population, and a post on depression may be relevant to them.

Comment author: gjm 15 September 2017 03:39:37PM *  6 points [-]

A few comments:

Q10: misspelling: "monogomous" should be "monogamous"

Q27: not clear what "as a community initiative" means. My actual impression is that LW2.0 is a project a few people are working on, but that most of "the community" has little visibility of it or input into it.

Q31: really needs an "insufficient data for meaningful answer" option.

Q32: fails to distinguish between "this is an incredibly important goal" and "this is the only goal that matters". If I became 100% convinced that no version of LW was ever going to raise the general sanity waterline I would be disappointed (note: I am already somewhat disappointed) but I wouldn't leave because I like being part of the community for other reasons too.

Q33: totally unclear what it's asking, and it seems like there's far too little information to give an answer that means anything. What is "this forum"? Specifically cafechesscourt (about which I know nothing), or is the picture just there to add colour? The question seems to presuppose that I can tell whether a forum would be worth participating in given only the identity of the person running it, which is bonkers.

Q33ff: there seems to be an underlying assumption here that what LW primarily needs is a technically better forum. I think the problems LW has faced have mostly been social rather than technical, and the main impact technical problems have had is that they have made it harder to address some social issues. (For instance, it was difficult to react appropriately to sustained malfeasance by a political fanatic because the LW server was (a) under the control of people who had little stake in its success and (b) running code that was painful to work with. But the actual problem was the political fanatic's behaviour.)

Q36: shouldn't forbid comments on unchosen options. Don't you want to know why someone doesn't care about something that sounds like it might matter?

Mental health questions: Might be worth distinguishing between "confident I would get a formal diagnosis if examined" and "think I probably have something similar". (The depression question 51, if you pick either version of "yes", opens up a question 52 asking whether you "still qualify for a depression diagnosis". If that distinction doesn't matter, maybe it should say something more like "still suffer from depression". Incidentally, I don't know why that question does this but the others don't. I guess depression more often goes away completely than the others.)

Q67: for some of these, some people with sufficient natural aptitude might have attained a high level of skill without the extreme effort implied by the rightmost column. (E.g., I'm a professional mathematician; I never "practised calculus every day" so far as I can recall.) I am guessing that in such cases you still want that rightmost column ticked (and maybe I should consider effort on more difficult branches of mathematics related to calculus to qualify) but it's not entirely clear. It's also unclear how broadly to take some of the categories; e.g., if someone has put a lot of effort into musical performance, does that count as "painting/drawing/etc."? (Depends whether "etc." is meant to cover other artistic endeavours or other forms of visual art or what.)

Q71,72: I wonder whether you want one more question, asking to what extent people think research into friendly AI is important. (So then you have a three-level funnel: worth working on friendly AI? worth doing it mathematically? is MIRI doing that well? rather than a two-level funnel.)

Q92: is kinda meaningless without knowing what country the respondent is from. (At least in so far as their opinion on this is an actual opinion rather than a mere party label.)

Q100ff: I think the biggest barriers to feasibility of replacing humans in many occupations, even discounting cases where being nonhuman is as such an obstacle, will likely not be down to intelligence unless that's broadened a great deal. Consider, e.g., opera singers or prostitutes. Machines trying to do these jobs will need to look and sound human not only because of people saying "ewwww, fake person" but also because the jobs intrinsically require specifically human-like performance.

Q108: crying out for a probability estimate rather than yes/no. I suspect most people signed up for cryonics think it probably won't work.

Q115ff: unclear (and I think it matters a lot for many people's opinions and feelings) what sort of modification is envisaged and how reliably it's supposed to be known to work. E.g., I have a tweenaged child and if someone said "here is a thing we can do that will increase her intelligence" then, quite apart from the fact that it would need to be her decision as well as mine, I would be incredibly skeptical and greatly disinclined to try the treatment for fear of unforeseen adverse consequences. And even if it were somehow known to be safe, not to mess up people's personalities, etc., it would feel like a big scary intervention in an existing person's life, whereas an in utero treatment might feel quite different. I would guess that this sort of thing will only (at least in the foreseeable future) be feasible in utero, or perhaps even in vitro before implantation, but the question doesn't make it clear what hypothetical we're supposed to be entertaining. (And the reference to "your child" makes it hard for those who have children not to think in terms of modifications to the children they already have, however old.)

Q122: really needs a "don't know / no strong opinion / it varies" option.

Q124: really needs probabilities or something. I can totally envisage scenarios where "the end of work" is a big win and others where it's a huge disaster.

Q127: really wants to be accompanied by some sort of estimate of how probable the event is. If someone thinks there's a 10^-6 chance of any of these things, it doesn't matter much which they think most likely.

Comment author: ingres 15 September 2017 08:37:27PM *  1 point [-]

Thanks for your feedback. While I agree with you at many (even most) points, there are several considerations to keep in mind:

  • It is not possible for me to change the questions at this point in time, There are nearly 200 responses at this point and it would be completely unfair of me to force everyone to retake the survey so I can fix most of the offered feedback. As a consequence I can only take these as potential improvements for the next survey.

  • The questions in the AI Progress section are ripped directly from an associated study, I have no control over their content or methodology besides replicating them as accurately and faithfully as I can.

  • Some simplification of scenarios is necessary to make them fit into a <150 question survey. There are also real limits to how much effort I can expect from people in terms of engaging with a scenario and that is why certain things such as the probability that a genetic treatment will be safe aren't included. Many things could use a probability but if I asked for one every time people would probably get frustrated with the survey and give up.

In the future we're looking at changing platforms so that the survey can be offered in a 'module' format which allows it to be taken in chunks over a much longer period of time with more detail in individual sections.

  • Quite often it is useful for a survey to explicitly not include a neutral option. Picking between two things is mentally difficult and a neutral option offers a path of least resistance which people are more likely to click than put in the effort even if that doesn't represent their true opinion.

  • At analysis time I have access to a respondents previous answers.

  • Our survey software prevents me from doing certain things which would be desirable.

Responding to certain specific questions:

Q33: Yes, CafeChesscourt. The question is "given nothing but the appearance of this forum" (CafeChesscourt put approximately zero effort into software, making it a useful control about how important software is to a forums success), would you use it if one of these "celebrity users" listed below were someone that ran it and used it regularly? The purpose of this question is to gauge how useful it is to get people who are popular to endorse a discussion forum. And while you might think it's bonkers to choose a forum that way, empirically many people will show up to places that are often reviled like Tumblr if the right person asks them to.

Q33ff: I agree that is an assumption many people are making in regards to this issue, that assumption is not there in the question however as its purpose was me teasing at the idea that perhaps focusing on technical excellence is the wrong metric.

Q36: I believe this is a limitation of the software, though I'll go ahead and double check since letting people write stuff in wouldn't materially effect the survey results.

Q52: Depression does go away more often than the others. But more importantly on the 2016 survey we had somewhat horrifying rates of depression. It was difficult to distinguish if this was because everybody gets depression at least once and then 'gets over it' or because that many LessWrongers are actually depressed. This question lets us tell the difference.

Q67: Yes. As you can tell this is a hard question to ask people but I think it's important enough to be worth asking even if imperfectly. In a future survey Calculus could probably be changed to "Calculus or 'higher maths'". The painting/etc option is meant to apply to visual arts. In the future it would probably be better to more accurately specify. In the grand scheme of things it will probably not spoil this years survey results to have the occasional musician pick it under the impression it counts. (I should also add music, thanks for the tip.)

Update (Fri Sep 15 13:46:59 PDT 2017): The issue in question 36 turned out to be fixable, thanks for the help!

Comment author: gjm 18 September 2017 08:33:38AM 0 points [-]

For the avoidance of doubt, I appreciate your "several considerations" and in particular was not suggesting that you should hack the questions about mid-survey. And, er, I realise that I just posted a bunch of criticism without adding: thank you very much for doing the survey; I think it will be interesting and useful; the fact that I have some quibbles doesn't make that any less true. So please consider that added :-).

As a single data point, on Q33 I attempted to answer as if the question meant something like "If all you knew about a new forum was that X was running the show, would you be likely to check it out for that reason?" on the grounds that that was the most non-bonkers interpretation I could give the question. If it was meant to be more like "If X was running the show, and the forum had no other merits, would X's leadership be enough to make you use it and stick around despite its lack of other merits?" then my answer, at least, will not be informative. I suspect I'm not alone :-).

Comment author: morganism 14 September 2017 08:32:44PM 2 points [-]

can't access "font fort" with noScript on. It needs to be expressed in the link html, so i can whitelist it.

Comment author: SaidAchmiz 15 September 2017 01:46:43AM 2 points [-]

Hi, could you elaborate on what you mean by "It needs to be expressed in the link html"?

Comment author: efenj 13 September 2017 09:36:19PM 1 point [-]

Firstly, thank you for the survey and for the option of exporting one's answers!

Questions that I found ambiguous or without a clear, correct answer (for future reference, since changing the survey midway is a terrible idea):

  1. Is it fundamentally important to you that the 'rationality movement' ever produces a measurable increase in general sanity? (i.e, if you were shown conclusive proof it will not you would likely leave)?

What do you answer if you believe that it is fundamentally important, and worth trying, but still unlikely to succeed (i.e. we're probably doomed, but we should still make an effort)?

  1. Do you attend Less Wrong meetups? Yes, once or a few times

Attended once or a few times, in total, or attend once or a few times per year/other reasonable time period?

Comment author: ingres 13 September 2017 09:43:49PM 2 points [-]
  1. Yes.

  2. Total, I would think.

Comment author: efenj 14 September 2017 02:00:42AM 1 point [-]

Thanks for the very fast reply!

I interpreted 2 correctly (in line with your reading), for 1, the "you would likely leave" part misled me.

Comment author: lifelonglearner 13 September 2017 09:06:40PM 5 points [-]

Re: The SAT question. The SAT's actually reverted back to a score out of 1600. Slight nitpick, given that you mention that the current status quo is out of 2400.

Comment author: gbear605 13 September 2017 05:57:16PM 4 points [-]

For being diagnosed with depression, do you include only major depressive disorder or do you also include persistent depressive disorder and adjustment disorder?

Comment author: ingres 13 September 2017 06:52:20PM 0 points [-]

Any form.

Comment author: Dagon 13 September 2017 02:40:33PM 5 points [-]

Took most of the survey, but seem to have submitted early somehow.

Comment author: pepe_prime 13 September 2017 01:20:21PM 10 points [-]

[Survey Taken Thread]

By ancient tradition, if you take the survey you may comment saying you have done so here, and people will upvote you and you will get karma.

Let's make these comments a reply to this post. That way we continue the tradition, but keep the discussion a bit cleaner.

Comment author: marchdown 23 September 2017 02:38:49AM 3 points [-]

Survey taken.

Comment author: alexey 22 September 2017 08:34:26PM 3 points [-]

I've taken the survey.

Comment author: kvas 21 September 2017 12:35:04PM 3 points [-]

I took the survey. It was long but fun. Thanks for the work you've put into designing it and processing the results.

Comment author: plethora 21 September 2017 09:54:16AM 4 points [-]

I have taken the survey.

Comment author: jkadlubo 21 September 2017 08:24:30AM 4 points [-]

I've taken the survey. Possibly my first activity here this year

Comment author: tenthkrige 20 September 2017 01:55:40PM 7 points [-]

I have also taken the survey.

Comment author: Viliam 19 September 2017 09:47:00PM 7 points [-]

Done.

Comment author: scarcegreengrass 19 September 2017 03:59:44PM 8 points [-]

I took the survey. It's probably my favorite survey of each year :) Thanks.

Comment author: fortyeridania 18 September 2017 05:37:45AM 9 points [-]

Done.

Comment author: EricHerboso 18 September 2017 03:52:32AM 9 points [-]

I have taken the survey.

Comment author: iceman 17 September 2017 09:05:29PM 11 points [-]

Survey taken!

Comment author: eukaryote 17 September 2017 07:46:04AM 17 points [-]

I have taken the survey, please shower me in karma.

Comment author: J_Thomas_Moros 17 September 2017 04:26:32AM 17 points [-]

I have completed the survey and upvoted everyone else on this thread

Comment author: blob 16 September 2017 12:42:55PM 16 points [-]

I've taken the survey.

Comment author: SolveIt 15 September 2017 11:18:52PM 17 points [-]

I have taken the survey!

Comment author: gjm 15 September 2017 03:06:38PM *  18 points [-]

I either have taken the survey in the last couple of hours, or have begun taking it and hereby commit to submitting a completed survey in the next couple of hours.

(Expressed in this form so as not to leak information about which survey is whose.)

[EDITED to add:] This is just to confirm that I have in fact completed the survey. I have also upvoted all the other taken-the-survey comments currently present.

Comment author: DragonGod 15 September 2017 02:34:13PM 19 points [-]

I took the survey, phew that was long. I added a public key for what it's worth.

Comment author: Artaxerxes 15 September 2017 01:13:55PM 18 points [-]

I did it!

Comment author: Regex 15 September 2017 05:22:47AM 18 points [-]

I have taken the survey... away from everyone.

No one can have it.

It lives under my bed now.

Comment author: richard_reitz 15 September 2017 04:13:03AM 18 points [-]

I have taken the survey.

Comment author: moridinamael 14 September 2017 08:41:49PM 19 points [-]

It is done.

Comment author: Zvi 14 September 2017 05:11:48PM 21 points [-]

I have taken the survey.

Comment author: Bitnotri 14 September 2017 05:03:18PM 20 points [-]

Taken

Comment author: Gyrodiot 14 September 2017 03:54:12PM 18 points [-]

I have taken the survey. Cannot remember my old public key, shrug

Comment author: RainbowSpacedancer 14 September 2017 03:34:57PM 20 points [-]

I have taken the survey.

Comment author: SoerenE 14 September 2017 01:08:51PM 20 points [-]

I have taken the survey.

Comment author: JenniferRM 14 September 2017 10:40:31AM 21 points [-]

I took the survey and upvoted every comment already here.

Comment author: maxjmartin 14 September 2017 09:44:50AM 21 points [-]

Survey completed

Comment author: enye-word 14 September 2017 05:33:34AM 22 points [-]

I have also taken this survey.

Comment author: Lark 14 September 2017 12:54:45AM 21 points [-]

Done!

Comment author: LawrenceC 14 September 2017 12:41:03AM 21 points [-]

I took the survey!

Comment author: g_pepper 13 September 2017 10:30:36PM 21 points [-]

I have taken the survey.

Comment author: lifelonglearner 13 September 2017 09:06:53PM 21 points [-]

I took the survey!

Comment author: RowanE 13 September 2017 07:00:52PM 21 points [-]

I have taken the survey.

Comment author: ingres 13 September 2017 06:52:39PM 21 points [-]

I have taken the survey.

Comment author: gbear605 13 September 2017 06:36:31PM 21 points [-]

I took the survey. I feel like the questions that ask for numeric answers about the probability of AI risk should have been optional because I have very weak fews about them

Comment author: ingres 13 September 2017 06:51:35PM 3 points [-]

Every question is optional besides the first two.

Comment author: adamzerner 13 September 2017 05:41:28PM 21 points [-]

Taken!

Comment author: NancyLebovitz 13 September 2017 04:32:29PM 22 points [-]

I have taken the survey.

Comment author: Screwtape 13 September 2017 03:13:36PM 22 points [-]

I have taken the survey. I don't remember there being a public key question at the end of the last one though, which is a shame since that obviously means I don't remember what mine was last time.

Comment author: ingres 13 September 2017 07:00:55PM *  0 points [-]

There wasn't. There was one on the 2014 survey. It's one of the missing items I decided to add back in.

Comment author: Elo 13 September 2017 01:27:19PM 23 points [-]

I took the survey. Towards a better future!

Comment author: pepe_prime 13 September 2017 01:20:36PM 23 points [-]

I have taken the survey.

Comment author: Aharon 13 September 2017 11:44:08AM 7 points [-]

Thanks for your work in creating the survey, and for LesserWrong. I shared the link to the survey in our meet-up group, and hope many people will contribute.

Comment author: Thomas 13 September 2017 10:18:02AM 3 points [-]

Previous session is set to be finished.

Your browser reports that it was used previously to answer this survey. We are resetting the session so that you can start from the beginning.

Click here to start the survey.

I have just pressed Enter after my country's name. Fix this!

Comment author: ingres 13 September 2017 06:51:59PM 0 points [-]

I think I'm going to need some more information. Can't fix a bug I can't reproduce.

Comment author: Thomas 13 September 2017 07:53:51PM 0 points [-]

My country isn't from the list of the default choices. So I type it and pressed Enter. It's all I remember.

Comment author: ingres 14 September 2017 06:37:28AM 2 points [-]

This issue should be fixed now, thanks for your report.

Comment author: ingres 13 September 2017 08:48:13PM 0 points [-]

That helps, thank you.

Comment author: ingres 13 September 2017 07:01:55AM *  2 points [-]

In addition to the email listed at the end of the survey, you can report bugs by replying to this post and I'll keep a log of their resolution:

Wed Sep 13 00:00:55 PDT 2017: Fixed bug where yourmorals.org survey links took user away from survey page instead of opening in new tab. (If this issue resurfaces please let me know.)

I'm also going to go through and fix the same issue in as many other survey links as I can. In the meantime work around by right clicking to open.

Wed Sep 13 21:50:11 PDT 2017: Fixed bug where enter key on text forms would throw out your survey information. Multiple people reported this and it should no longer happen.

Fri Sep 15 13:46:59 PDT 2017: Fixed bug that prevented survey users from commenting on boxes they hadn't checked in question 36.