To arms, my brothers and sisters! This could be a highly consequential opportunity for those of us with concerns but without the connections or finesse or approach them independently with those concerns! Upvote critical questions!

Here is a thread from LessWrong/Reddit user 'jimrandomh' asking questions which will be most pertinent to the LessWrong userbase, and supporters of MIRI.

Here is a long thread where Eliezer himself jumps in and has an interesting conversation attempting to bridge gaps of understanding between the perception of other Machine Learning (ML) researchers of MIRI's goals and concerns, vs. what their goals and concerns, re: AI safety, actually are.

Thank you.

I could ban that account. Thoughts about whether that would be a good idea?

Religion solves some coordination problems very well. Witness religions outlasting numerous political and philosophical movements, often through coordinated effort. Some wrong beliefs assuage bad emotions and thoughts, allowing humans to internally deal with the world beyond the reach of god. Some of the same wrong beliefs also hurt and kill a shitload of people, directly and indirectly.

My personal belief is that religions were probably necessary for humanity to rise from agricultural to technological societies, and tentatively necessary to maintain technological societies until FAI, especially in a long-takeoff scenario. We have limited evidence that religion-free or wrong-belief-free societies can flourish. Most first-world nations are officially and practically agnostic but have sizable populations of religious people. The nations which are actively anti-religious generally have their own strong dogmatic anti-scientific beliefs that the leaders are trying to push, and they still can't stomp out all religions.

Basically, until doctors can defeat virtually all illness and death and leaders can effectively coordinate global humane outcomes without religions I think that religions serve as a sanity line above destructive hedonism or despair.

I will run it if no one else has piped up. Scheduled for Feb if no one else takes it on. I will post asking for questions in a month.

Notes

1 Based on my current impressions of changing and growing cause selections within effective altruism, here are some other causes I believe it would've been useful to include this year, and perhaps can be added to next year's survey.

• Animal welfare (industrial farming)
• Animal welfare (other)
• Existential risks (biosecurity/biotechnology)

2 Based on my current impressions of the growing number of major effective altruism organizations, here are some other organizations I believe it would've been useful to include this year as sources for first hearing about effective altruism, and perhaps can be added to next year's survey.

• Charity Science
• Center For Applied Rationality
• Effective Altruism Foundation (formerly GBS Schweiz)
• The Gates Foundation
• Peter Singer's online Coursera course
• University or college course

3 Based on my current impressions of the growing number of major effective altruism organizations, and also their recommended charities, I expect it would make sense to include in the list of charities asked if donated to this year, and perhaps can be included next year:

• The full list of Givewell's recommended and standout charities
• The full list of ACE's recommended and standout charities
• The full list of TLYCS's and GWWC's recommended charities
• Charity Science
• The Effective Altruism Foundation and its affiliate organizations
• The Centre for Effective Altruism and its affiliate organizations

Really, of all the lists which don't have enough options this year, this one is the most lacking. Note: there has been rapid change to the effective altruism community as of late, with an unusually rapid expansion in the number of organizations recognized as effective in 2015 than in prior years. Thus, I consider it understandable the team running the EA Survey this year hasn't been able to keep up pace with the growing and full list of organizations favored by the broader effective altruism community.

4 For education levels, between "High school (and lower)" and "Undergraduate degree", I'd include "2-year college/Associate's degree" as another option.

5 For the question "What broad career are you planning to follow?", I believe it would've been useful to include the following options in this year's survey, and perhaps can be added to next year's survey:

• social entrepreneurship
• advocacy
• policy work

That's all. Otherwise, assume anything else I didn't comment on is free or any errors or conspicuous omissions (I could notice). Great work!

I see that you can read my mind and my votes. Glad you have that ability

It's part of the Dark Arts package. I'll dryly observe that I knew you downvoted him - how do you think I knew that you downvoted him? It's not like downvotes come with names attached. Yes, I can "read your mind", which is to say, I read the -massive- amounts of connotation information associated with otherwise bland text.

Can you please provide evidence of what I am thinking and how I am voting?

You, uh, admitted to it? "I thought his comments were not worth attention"

I have not a clue whether this sort of marketing is a good idea. Let me be clear what I mean: I think there's maybe a 30-40% chance that Gleb is having a net positive impact through these outreach efforts. I also think there's maybe a 10-20% chance that he's having a horrific long-term negative impact through these outreach efforts. Thus the whole thing makes me uncomfortable.

So here's some of the concerns I see; I've gone to some effort to be fair to Gleb, and not assume anything about his thoughts or motivations:

• By presenting these ideas in weakened forms (either by giving short or invalid argumentation, or putting it in venues or contexts with negative associations), he may be memetically immunizing people against the stronger forms of the ideas.
• By teaching people using arguments from authority, he may be worsening the primary "sanity waterline" issues rather than improving them. The articles, materials, and comments I've seen make heavy use of language like "science-based", "research-based" and "expert". The people reading these articles in general have little or no skill at evaluating such claims, so that they effectively become arguments from authority. By rhetorically convincing them to adopt the techniques or thoughts, he's spreading quite possibly helpful ideas, but reinforcing bad habits around accepting ideas.
• Gleb's writing style strikes me as very unauthentic feeling. Let me be clear I don't mean to accuse him of anything negative; but I intuitively feel a very negative reaction to his writing. It triggers emotional signals in me of attempted deception and rhetorical tricks (whether or not this is his intent!) His writing risks associating "rationality" with such signals (should other people share my reactions) and again causing immunization, or even catalyzing opposition.

An illustration of the nightmare scenario from such an outreach effort would be that, 3 years from now when I attempt to talk to someone about biases, they respond by saying "Oh god don't give me that '6 weird tips' bullshit about 'rational thinking', and spare me your godawful rhetoric, gtfo."

Like I said at the start, I don't know which way it swings, but those are my thoughts and concerns. I imagine they're not new concerns to Gleb. I still have these concerns after reading all of the mitigating argumentation he has offered so far, and I'm not sure of a good way to collect evidence about this besides running absurdly large long-term "consumer" studies.

I do imagine he plans to continue his efforts, and thus we'll find out eventually how this turns out.

I found out about Omnilibrium a couple months ago, and I was thinking of joining in eventually. I was also thinking of telling some friends of mine who might want to get in on it even more than I do about it. However, I've been thinking if I told lots of people, or they themselves told lots of people, then suddenly Omnilibrium might get flooded with dozens of new users at once. I don't know how big that is compared to the whole community, but I was thinking Omnilibrium would be averse to it growing it too big, as well-kempt gardens die by pacificism and all that. But then, Slate Star Codex linked to it a few weeks ago. So, that's plausibly hundreds of new users flooding it.

I'm wondering, how do the admins of Omnilibrium feel about this? Are you happy to have many new users? Are you upset SSC linked to Omnilibrium, bringing it to the attention of so many people who may not necessarily maintain the quality of discourse current users of Omnilibrium have gotten accustomed to?