Contents

• Summary
• Background
• Previous surveys
• Motivation
• Survey questions for the first survey
• Survey questions for the second survey
• Survey questions for the third survey (Google Consumer Surveys)
• Results
  • S1Q1: number of Wikipedia pages read per week
  • S1Q2: affinity for Wikipedia in search results
  • S1Q3: section vs whole page
  • S1Q4: search functionality on Wikipedia and surprise at lack of Wikipedia pages
  • S1Q5: behavior on pages
  • S2Q1: number of Wikipedia pages read per week
  • S2Q2: multiple-choice of articles read
  • S2Q3: free response of articles read
  • S2Q4: free response of surprise at lack of Wikipedia pages
  • S3Q1 (Google Consumer Surveys)
• Summaries of responses (exports for SurveyMonkey, weblink for Google Consumer Surveys)
• Survey-making lessons
• Further questions
• Further reading
• Acknowledgements
• Document source and versions
• License

Summary

The summary is not intended to be comprehensive. It highlights the most important takeaways you should get from this post.

• Vipul Naik and I are interested in understanding how people use Wikipedia. One reason is that we are getting more people to work on editing and adding content to Wikipedia. We want to understand the impact of these edits, so that we can direct efforts more strategically. We are also curious!

• From May to July 2016, we conducted two surveys of people’s Wikipedia usage. We collected survey responses from audience segments include Slate Star Codex readers, Vipul’s Facebook friends, and a few audiences through SurveyMonkey Audience and Google Consumer Surveys. Our survey questions measured how heavily people use Wikipedia, what sort of pages they read or expected to find, the relation between their search habits and Wikipedia, and other actions they took within Wikipedia.

• Different audience segments responded very differently to the survey. Notably, the SurveyMonkey audience (which is closer to being representative of the general population) appears to use Wikipedia a lot less than Vipul’s Facebook friends and Slate Star Codex readers. Their consumption of Wikipedia is also more passive: they are less likely to explicitly seek Wikipedia pages when searching for a topic, and less likely to engage in additional actions on Wikipedia pages. Even the college-educated SurveyMonkey audience used Wikipedia very little.

• This is tentative evidence that Wikipedia consumption is skewed towards a certain profile of people (and Vipul’s Facebook friends and Slate Star Codex readers sample much more heavily from that profile). Even more tentatively, these heavy users tend to be more “elite” and influential. This tentatively led us to revise upward our estimates of the social value of a Wikipedia pageview.

• This was my first exercise in survey construction. I learned a number of lessons about survey design in the process.

• All the survey questions, as well as the breakdown of responses for each of the audience segments, are described in this post. Links to PDF exports of response summaries are at the end of the post.

Background

At the end of May 2016, Vipul Naik and I created a Wikipedia usage survey to gauge the usage habits of Wikipedia readers and editors. SurveyMonkey allows the use of different “collectors” (i.e. survey URLs that keep results separate), so we circulated several different URLs among four locations to see how different audiences would respond. The audiences were as follows:

• SurveyMonkey’s United States audience with no demographic filters (62 responses, 54 of which are full responses)
• Vipul Naik’s timeline (post asking people to take the survey; 70 responses, 69 of which are full responses). For background on Vipul’s timeline audience, see his page on how he uses Facebook.
• The Wikipedia Analytics mailing list (email linking to the survey; 7 responses, 6 of which are full responses). Note that due to the small size of this group, the results below should not be trusted, unless possibly when the votes are decisive.
• Slate Star Codex (post that links to the survey; 618 responses, 596 of which are full responses). While Slate Star Codex isn’t the same as LessWrong, we think there is significant overlap in the two sites’ audiences (see e.g. the recent LessWrong diaspora survey results).
• In addition, although not an actual audience with a separate URL, several of the tables we present below will include an “H” group; this is the heavy users group of people who responded by saying they read 26 or more articles per week on Wikipedia. This group has 179 people: 164 from Slate Star Codex, 11 from Vipul’s timeline, and 4 from the Analytics mailing list.

We ran the survey from May 30 to July 9, 2016 (although only the Slate Star Codex survey had a response past June 1).

After we looked at the survey responses on the first day, Vipul and I decided to create a second survey to focus on the parts from the first survey that interested us the most. The second survey was only circulated among SurveyMonkey’s audiences: we used SurveyMonkey’s US audience with no demographic filters (54 responses), as well as a US audience of ages 18–29 with a college or graduate degree (50 responses). We first ran the survey on the unfiltered audience again because the wording of our first question was changed and we wanted to have the new baseline. We then chose to filter for young college-educated people because our prediction was that more educated people would be more likely to read Wikipedia (the SurveyMonkey demographic data does not include education, and we hadn’t seen the Pew Internet Research surveys in the next section, so we were relying on our intuition and some demographic data from past surveys) and because young people in our first survey gave more informative free-form responses in survey 2 (SurveyMonkey’s demographic data does include age).

We ran a third survey on Google Consumer Surveys with a single question that was a word-to-word replica of the first question from the second survey. The main motivation here was that on Google Consumer Surveys, a single-question survey costs only 10 cents per response, so it was possible to get to a large number of responses at relatively low cost, and achieve more confidence in the tentative conclusions we had drawn from the SurveyMonkey surveys.

Previous surveys

Several demographic surveys regarding Wikipedia have been conducted, targeting both editors and users. The surveys we found most helpful were the following:

• The 2010 Wikipedia survey by the Collaborative Creativity Group and the Wikimedia Foundation. The explanation before the bottom table on page 7 of the overview PDF has “Contributors show slightly but significantly higher education levels than readers”, which provides weak evidence that more educated people are more likely to engage with Wikipedia.
• The Global South User Survey 2014 by the Wikimedia Foundation
• Pew Internet Research’s 2011 survey: “Education level continues to be the strongest predictor of Wikipedia use. The collaborative encyclopedia is most popular among internet users with at least a college degree, 69% of whom use the site.” (page 3)
• Pew Internet Research’s 2007 survey

Note that we found the Pew Internet Research surveys after conducting our own two surveys (and during the write-up of this document).

Motivation

Vipul and I ultimately want to get a better sense of the value of a Wikipedia pageview (one way to measure the impact of content creation), and one way to do this is to understand how people are using Wikipedia. As we focus on getting more people to work on editing Wikipedia – thus causing more people to read the content we pay and help to create – it becomes more important to understand what people are doing on the site.

For some previous discussion, see also Vipul’s answers to the following Quora questions:

• What are the various parameters that affect the value of a pageview?
• What’s the relative social value of 1 Quora pageview (as measured by Quora stats http://www.quora.com/stats) and 1 Wikipedia pageview (as measured at, say, Wikipedia article traffic statistics)?

Wikipedia allows relatively easy access to pageview data (especially by using tools developed for this purpose, including one that Vipul made), and there are some surveys that provide demographic data (see “Previous surveys” above). However, after looking around, it was apparent that the kind of information our survey was designed to find was not available.

I should also note that we were driven by our curiosity of how people use Wikipedia.

Survey questions for the first survey

For reference, here are the survey questions for the first survey. A dummy/mock-up version of the survey can be found here: https://www.surveymonkey.com/r/PDTTBM8.

The survey introduction said the following:

This survey is intended to gauge Wikipedia use habits. This survey has 3 pages with 5 questions total (3 on the first page, 1 on the second page, 1 on the third page). Please try your best to answer all of the questions, and make a guess if you’re not sure.

And the actual questions:

1. How many distinct Wikipedia pages do you read per week on average?

   • less than 1
   • 1 to 10
   • 11 to 25
   • 26 or more

2. On a search engine (e.g. Google) results page, do you explicitly seek Wikipedia pages, or do you passively click on Wikipedia pages only if they show up at the top of the results?

   • I explicitly seek Wikipedia pages
   • I have a slight preference for Wikipedia pages
   • I just click on what is at the top of the results

3. Do you usually read a particular section of a page or the whole article?

   • Particular section
   • Whole page

4. How often do you do the following? (Choices: Several times per week, About once per week, About once per month, About once per several months, Never/almost never.)

   • Use the search functionality on Wikipedia
   • Be surprised that there is no Wikipedia page on a topic

5. For what fraction of pages you read do you do the following? (Choices: For every page, For most pages, For some pages, For very few pages, Never. These were displayed in a random order for each respondent, but displayed in alphabetical order here.)

   • Check (click or hover over) at least one citation to see where the information comes from on a page you are reading
   • Check how many pageviews a page is getting (on an external site or through the Pageview API)
   • Click through/look for at least one cited source to verify the information on a page you are reading
   • Edit a page you are reading because of grammatical/typographical errors on the page
   • Edit a page you are reading to add new information
   • Look at the “See also” section for additional articles to read
   • Look at the editing history of a page you are reading
   • Look at the editing history solely to see if a particular user wrote the page
   • Look at the talk page of a page you are reading
   • Read a page mostly for the “Criticisms” or “Reception” (or similar) section, to understand different views on the subject
   • Share the page with a friend/acquaintance/coworker

For the SurveyMonkey audience, there were also some demographic questions (age, gender, household income, US region, and device type).

Survey questions for the second survey

For reference, here are the survey questions for the second survey. A dummy/mock-up version of the survey can be found here: https://www.surveymonkey.com/r/28BW78V.

The survey introduction said the following:

This survey is intended to gauge Wikipedia use habits. Please try your best to answer all of the questions, and make a guess if you’re not sure.

This survey has 4 questions across 3 pages.

In this survey, “Wikipedia page” refers to a Wikipedia page in any language (not just the English Wikipedia).

And the actual questions:

1. How many distinct Wikipedia pages do you read (at least one sentence of) per week on average?

   • Fewer than 1
   • 1 to 10
   • 11 to 25
   • 26 or more

2. Which of these articles have you read (at least one sentence of) on Wikipedia (select all that apply)? (These were displayed in a random order except the last option for each respondent, but displayed in alphabetical order except the last option here.)

   • Adele
   • Barack Obama
   • Bernie Sanders
   • China
   • Donald Trump
   • Google
   • Hillary Clinton
   • India
   • Japan
   • Justin Bieber
   • Justin Trudeau
   • Katy Perry
   • Taylor Swift
   • The Beatles
   • United States
   • World War II
   • None of the above

3. What are some of the Wikipedia articles you have most recently read (at least one sentence of)? Feel free to consult your browser’s history.

4. Recall a time when you were surprised that a topic did not have a Wikipedia page. What were some of these topics?

Survey questions for the third survey (Google Consumer Surveys)

This survey had exactly one question. The wording of the question was exactly the same as that of the first question of the second survey.

1. How many distinct Wikipedia pages do you read (at least one sentence of) per week on average?

   • Fewer than 1
   • 1 to 10
   • 11 to 25
   • 26 or more

One slight difference was that whereas in the second survey, the order of the options was fixed, the third survey did a 50/50 split between that order and the exact reverse order. Such splitting is a best practice to deal with any order-related biases, while still preserving the logical order of the options. You can read more on the questionnaire design page of the Pew Research Center.

Results

In this section we present the highlights from each of the survey questions. If you prefer to dig into the data yourself, there are also some exported PDFs below provided by SurveyMonkey. Most of the inferences can be made using these PDFs, but there are some cases where additional filters are needed to deduce certain percentages.

We use the notation “SnQm” to mean “survey n question m”.

S1Q1: number of Wikipedia pages read per week

Here is a table that summarizes the data for Q1:

Here are some highlights from the first question that aren’t apparent from the table:

• Of the people who read fewer than 1 distinct Wikipedia page per week (26 people), 68% were female even though females were only 48% of the respondents. (Note that gender data is only available for the SurveyMonkey audience.)

• Filtering for high household income ($150k or more; 11 people) in the SurveyMonkey audience, only 2 read fewer than 1 page per week, although most (7) of the responses still fall in the “1 to 10” category.

The comments indicated that this question was flawed in several ways: we didn’t specify which language Wikipedias count nor what it meant to “read” an article (the whole page, a section, or just a sentence?). One comment questioned the “low” ceiling of 26; in fact, I had initially made the cutoffs 1, 10, 100, 500, and 1000, but Vipul suggested the final cutoffs because he argued they would make it easier for people to answer (without having to look it up in their browser history). It turned out this modification was reasonable because the “26 or more” group was a minority.

S1Q2: affinity for Wikipedia in search results

We asked Q2, “On a search engine (e.g. Google) results page, do you explicitly seek Wikipedia pages, or do you passively click on Wikipedia pages only if they show up at the top of the results?”, to see to what extent people preferred Wikipedia in search results. The main implication to this for people who do content creation on Wikipedia is that if people do explicitly seek Wikipedia pages (for whatever reason), it makes sense to give them more of what they want. On the other hand, if people don’t prefer Wikipedia, it makes sense to update in favor of diversifying one’s content creation efforts while still keeping in mind that raw pageviews indicate that content will be read more if placed on Wikipedia (see for instance Brian Tomasik’s experience, which is similar to my own, or gwern’s page comparing Wikipedia with other wikis).

The following table summarizes our results.

One error on my part was that I didn’t include an option for people who avoided Wikipedia or did something else. This became apparent from the comments. For this reason, the “Just click on top results” options might be inflated. In addition, some comments indicated a mixed strategy of preferring Wikipedia for general overviews while avoiding it for specific inquiries, so allowing multiple selections might have been better for this question.

S1Q3: section vs whole page

This question is relevant for Vipul and me because the work Vipul funds is mainly whole-page creation. If people are mostly reading the introduction or a particular section like the “Criticisms” or “Reception” section (see S1Q5), then that forces us to consider spending more time on those sections, or to strengthen those sections on weak existing pages.

Responses to this question were fairly consistent across different audiences, as can be see in the following table.

Note that people were allowed to select more than one option for this question. The comments indicate that several people do a combination, where they read the introductory portion of an article, then narrow down to the section of their interest.

S1Q4: search functionality on Wikipedia and surprise at lack of Wikipedia pages

We asked about whether people use the search functionality on Wikipedia because we wanted to know more about people’s article discovery methods. The data is summarized in the following table.

Many people noted here that rather than using Wikipedia’s search functionality, they use Google with “wiki” attached to their query, DuckDuckGo’s “!w” expression, or some browser configuration to allow a quick search on Wikipedia.

To be more thorough about discovering people’s content discovery methods, we should have asked about other methods as well. We did ask about the “See also” section in S1Q5.

Next, we asked how often people are surprised that there is no Wikipedia page on a topic to gauge to what extent people notice a “gap” between how Wikipedia exists today and how it could exist. We were curious about what articles people specifically found missing, so we followed up with S2Q4.

Two comments on this question (out of 59) – both from the SSC group – specifically bemoaned deletionism, with one comment calling deletionism “a cancer killing Wikipedia”.

S1Q5: behavior on pages

This question was intended to gauge how often people perform an action for a specific page; as such, the frequencies are expressed in page-relative terms.

The following table presents the scores for each response, which are weighted by the number of responses. The scores range from 1 (for every page) to 5 (never); in other words, the lower the number, the more frequently one does the thing.

The table above provides a good ranking of how often people perform these actions on pages, but not the distribution information (which would require three dimensions to present fully). In general, the more common actions (scores of 2.5–4) had responses that clustered among “For some pages”, “For very few pages”, and “Never”, while the less common actions (scores above 4) had responses that clustered mainly in “Never”.

One comment (out of 43) – from the SSC group, but a different individual from the two in S1Q4 – bemoaned deletionism.

S2Q1: number of Wikipedia pages read per week

Note the wording changes on this question for the second survey: “less” was changed to “fewer”, the clarification “at least one sentence of” was added, and we explicitly allowed any language. We have also presented the survey 1 results for the SurveyMonkey audience in the corresponding rows, but note that because of the change in wording, the correspondence isn’t exact.

Comparing SM with S1SM, we see that probably because of the wording, the percentages have drifted in the direction of more pages read. It might be surprising that the young educated audience seems to have a smaller fraction of heavy users than the general population. However note that each group only had ~50 responses, and that we have no education information for the SM group.

S2Q2: multiple-choice of articles read

Our intention with this question was to see if people’s stated or recalled article frequencies matched the actual, revealed popularity of the articles. Therefore we present the pageview data along with the percentage of people who said they had read an article.

Below are four plots of the data. Note that r_s denotes Spearman’s rank correlation coefficient. Spearman’s rank correlation coefficient is used instead of Pearson’s r because the former is less affected by outliers. Note also that the percentage of respondents who viewed a page counts each respondent once, whereas the number of pageviews does not have this restriction (i.e. duplicate pageviews count), so we wouldn’t expect the relationship to be entirely linear even if the survey audiences were perfectly representative of the general population.

SM vs 2016 pageviews

SM vs 2015 pageviews

CEYP vs 2016 pageviews

CEYP vs 2015 pageviews

S2Q3: free response of articles read

The most common response was along the lines of “None”, “I don’t know”, “I don’t remember”, or similar. Among the more useful responses were:

• News stories (e.g. Death of Harambe, “WikiLeaks scandal” – unclear which page this is, since there are several pages on various aspects of WikiLeaks)
• Popular culture:
  • People including Megan Fox, LeBron James, Rita Hayworth
  • Works including Aladdin and the King of Thieves, X-Men: Apocalypse
• More traditional encyclopedic information (e.g. Emerald ash borer, Spain, Siphonophorae, Scolopendra gigantea)

S2Q4: free response of surprise at lack of Wikipedia pages

As with the previous question, the most common response was along the lines of “None”, “I don’t know”, “I don’t remember”, “Doesn’t happen”, or similar.

The most useful responses were classes of things: “particular words”, “French plays/books”, “Random people”, “obscure people”, “Specific list pages of movie genres”, “Foreign actors”, “various insect pages”, and so forth.

S3Q1 (Google Consumer Surveys)

The survey was circulated to a target size of 500 in the United States (no demographic filters), and received 501 responses.

Since there was only one question, but we obtained data filtered by demographics in many different ways, we present this table with the columns denoting responses and the rows denoting the audience segments. We also include the S1Q1SM, S2Q1SM, and S2Q1CEYP responses for easy comparison. Note that S1Q1SM did not include the “at least one sentence of” caveat. We believe that adding this caveat would push people’s estimates upward.

If you view the Google Consumer Surveys results online you will also see the 95% confidence intervals for each of the segments. Note that percentages in a row may not add up to 100% due to rounding or due to people entering “Other” responses. For the entire GCS audience, every pair of options had a statistically significant difference, but for some subsegments, this was not true.

We can see that the overall GCS data vindicates the broad conclusions we drew from SurveyMonkey data. Moreover, most GCS segments with a sufficiently large number of responses (50 or more) display a similar trend as the overall data. One exception is that younger audiences seem to be slightly less likely to use Wikipedia very little (i.e. fall in the “Fewer than 1” category), and older audiences seem slightly more likely to use Wikipedia very little.

Summaries of responses (exports for SurveyMonkey, weblink for Google Consumer Surveys)

SurveyMonkey allows exporting of response summaries. Here are the exports for each of the audiences.

• Survey 1, SurveyMonkey’s audience
• Survey 1, Vipul’s timeline
• Survey 1, Wikipedia Analytics mailing list
• Survey 1, Slate Star Codex
• Survey 1, Heavy users
• Survey 2, no demographic filters
• Survey 2, educated young people

The Google Consumer Surveys survey results are available online at https://www.google.com/insights/consumersurveys/view?survey=o3iworx2rcfixmn2x5shtlppci&question=1&filter=&rw=1.

Survey-making lessons

Not having any experience designing surveys, and wanting some rough results quickly, I decided not to look into survey-making best practices beyond the feedback from Vipul. As the first survey progressed, it became clear that there were several deficiencies in that survey:

• Question 1 did not specify what counts as reading a page.
• We did not specify which language Wikipedias we were considering (multiple people noted how they read other language Wikipedias other than the English Wikipedia).
• Question 2 did not include an option for people who avoid Wikipedia or do something else entirely.
• We did not include an option to allow people to release their survey results.

Further questions

The two surveys we’ve done so far provide some insight into how people use Wikipedia, but we are still far from understanding the value of Wikipedia pageviews. Some remaining questions:

• Could it be possible that even on non-obscure topics, most of the views are by “elites” (i.e. those with outsized impact on the world)? This could mean pageviews are more valuable than previously thought.
• On S2Q1, why did our data show that CEYP was less engaged with Wikipedia than SM? Is this a limitation of the small number of responses or of SurveyMonkey’s audiences?

Further reading

• “The great decline in Wikipedia pageviews (condensed version)” by Vipul Naik
• “In Defense Of Inclusionism” by gwern

Acknowledgements

Thanks to Vipul Naik for collaboration on this project and feedback while writing this document, and for supplying the summary section, and thanks to Ethan Bashkansky for reviewing the document. All imperfections are my own.

The writing of this document was sponsored by Vipul Naik. Vipul Naik also paid SurveyMonkey (for the cost of SurveyMonkey Audience) and Google Consumer Surveys.

Document source and versions

The source files used to compile this document are available in a GitHub Gist. The Git repository of the Gist contains all versions of this document since its first publication.

This document is available in the following formats:

• As an HTML file at http://lesswrong.com/r/discussion/lw/nru/wikipedia_usage_survey_results/
• As a PDF file at http://files.issarice.com/wikipedia-survey-results.pdf

License

This document is released to the public domain.

(Content note: A common suggestion for debiasing hindsight: try to think of many alternative historical outcomes. But thinking of too many examples can actually make hindsight bias worse.)

Followup to: Availability Heuristic Considered Ambiguous

Related to: Hindsight Bias

I.

Hindsight bias is when people who know the answer vastly overestimate its predictability or obviousness, compared to the estimates of subjects who must guess without advance knowledge.  Hindsight bias is sometimes called the I-knew-it-all-along effect.

The way that this bias is usually explained is via the availability of outcome-related knowledge. The outcome is very salient, but the possible alternatives are not, so the probability that people claim they would have assigned to an event that has already happened gets jacked up. It's also known that knowing about hindsight bias and trying to adjust for it consciously doesn't eliminate it.

This means that most attempts at debiasing focus on making alternative outcomes more salient. One is encouraged to recall other ways that things could have happened. Even this merely attenuates the hindsight bias, and does not eliminate it (Koriat, Lichtenstein, & Fischhoff, 1980; Slovic & Fischhoff, 1977).

II.

Remember what happened with the availability heuristic when we varied the number of examples that subjects had to recall? Crazy things happened because of the phenomenal experience of difficulty that recalling more examples caused within the subjects.

You might imagine that, if you recalled too many examples, you could actually make the hindsight bias worse, because if subjects experience alternative outcomes as difficult to generate, then they'll consider the alternatives less likely, and not more.

Relatedly, Sanna, Schwarz, and Stocker (2002, Experiment 2) presented participants with a description of the British–Gurkha War (taken from Fischhoff, 1975; you should remember this one). Depending on conditions, subjects were told either that the British or the Gurkha had won the war, or were given no outcome information. Afterwards, they were asked, “If we hadn’t already told you who had won, what would you have thought the probability of the British (Gurkhas, respectively) winning would be?”, and asked to give a probability in the form of a percentage.

Like in the original hindsight bias studies, subjects with outcome knowledge assigned a higher probability to the known outcome than subjects in the group with no outcome knowledge. (Median probability of 58.2% in the group with outcome knowledge, and 48.3% in the group without outcome knowledge.)

Some subjects, however, were asked to generate either 2 or 10 thoughts about how the outcome could have been different. Thinking of 2 alternative outcomes slightly attenuated hindsight bias (median down to 54.3%), but asking subjects to think of 10 alternative outcomes went horribly, horribly awry, increasing the subjects' median probability for the 'known' outcome all the way up to 68.0%!

It looks like we should be extremely careful when we try to retrieve counterexamples to claims that we believe. If we're too hard on ourselves and fail to take this effect into account, then we can make ourselves even more biased than we would have been if we had done nothing at all.

III.

But it doesn't end there.

Like in the availability experiments before this, we can discount the informational value of the experience of difficulty when generating examples of alternative historical outcomes. Then the subjects would make their judgment based on the number of thoughts instead of the experience of difficulty.

Just before the 2000 U.S. presidential elections, Sanna et al. (2002, Experiment 4) asked subjects to predict the percentage of the popular vote the major candidates would receive. (They had to wait a little longer than they expected for the results.)

Later, they were asked to recall what their predictions were.

Control group subjects who listed no alternative thoughts replicated previous results on the hindsight bias.

Experimental group subjects who listed 12 alternative thoughts experienced difficulty and their hindsight bias wasn't made any better, but it didn't get worse either.

(It seems the reason it didn't get worse is because everyone thought Gore was going to win before the election, and for the hindsight bias to get worse, the subjects would have to incorrectly recall that they predicted a Bush victory.)

Other experimental group subjects listed 12 alternative thoughts and were also made to attribute their phenomenal experience of difficulty to lack of domain knowledge, via the question: "We realize that this was an extremely difficult task that only people with a good knowledge of politics may be able to complete. As background information, may we therefore ask you how knowledgeable you are about politics?" They were then made to provide a rating of their political expertise and to recall their predictions.

Because they discounted the relevance of the difficulty of recalling 12 alternative thoughts, attributing it to their lack of political domain knowledge, thinking of 12 ways that Gore could have won introduced a bias in the opposite direction! They recalled their original predictions for a Gore victory as even more confident than they actually, originally were.

We really are doomed.

Hey, everyone! I want to share with you a project I've been working on for a while - http://rationalfiction.io.

I want it to become the perfect place to publish, discover, and discuss rational fiction.

We already have a lot of awesome stories, and I invite you to join and post more! =)

Yale Assistant Professor of Political Science Allan Dafoe is seeking Research Assistants for a project on the political dimensions of the existential risks posed by advanced artificial intelligence. The project will involve exploring issues related to grand strategy and international politics, reviewing possibilities for social scientific research in this area, and institution building. Familiarity with international relations, existential risk, Effective Altruism, and/or artificial intelligence are a plus but not necessary. The project is done in collaboration with the Future of Humanity Institute, located in the Faculty of Philosophy at the University of Oxford. There are additional career opportunities in this area, including in the coming academic year and in the future at Yale, Oxford, and elsewhere. If interested in the position, please email allan.dafoe@yale.edu with a copy of your CV, a writing sample, an unofficial copy of your transcript, and a short (200-500 word) statement of interest. Work can be done remotely, though being located in New Haven, CT or Oxford, UK is a plus.

A paper published in Astrobiology: A New Empirical Constraint on the Prevalence of Technological Species in the Universe (PDF), A. Frank and W.T. Sullivan.

From the abstract:

Recent advances in exoplanet studies provide strong constraints on all astrophysical terms in the Drake equation. [...] We find that as long as the probability that a habitable zone planet develops a technological species is larger than ~ 10^-24, humanity is not the only time technological intelligence has evolved.

They say we now know with reasonable certainty the total number of stars ever to exist (in the observable universe), and the average number of planets in the habitable zone. But we still don't know the probabilities of life, intelligence, and technology arising. They call this cumulative unknown factor f_bt.

Their result: for technological civilization to arise no more than once, with probability 0.01, in the lifetime of the observable universe, f_bt should be no greater than ~ 2.5 x 10^-24.

Discussion

It's convenient that they calculate the chance technological civilization ever arose, rather than the chance one exists now. This is just the number we need to estimate the likelihood of a Great Filter.

They state their result as "[if we set f_bt ≤ 2.5 x 10^-24, then] at in a statistical sense were we to rerun the history of the Universe 100 times, only once would a lone technological species occur". But I don't know what rerunning the Universe means. I also can't formulate this as saying "if we hadn't already observed the Universe to be apparently empty of life, we would expect it to contain or to have once contained life with a probability of 10²⁴", because that would ignore the chance that another civilization (if it counterfactually existed) would have affected or prevented the rise of life on Earth. Can someone help reformulate this? 

I don't know if their modern values for star and planet formation have been used in previous discussions of the Fermi paradox or the Great Filter. (The papers they cite for their values date from 2012, 2013 and 2015.) I also don't know if these values should be trusted, or what concrete values had been used previously. People on top of the Great Filter discussion probably already updated when the astronomical data came in.

Recap: (See also: http://lesswrong.com/r/discussion/lw/nef/the_thyroid_madness_core_argument_evidence/ and previous posts)

Chronic Fatigue Syndrome and Fibromyalgia all look far too much like the classical presentation of hypothyroidism for comfort, but thyroid hormone blood tests are normal.

Many alternative medicine practitioners, most prominently John Lowe, and several conventional medical doctors, most prominently Kenneth Blanchard, a practising endocrinologist with a longstanding practice completely free of lawsuits, have tried diagnosing hypothyroidism 'by clinical symptoms', and treating it with various combinations of thyroid hormones, and they all report success, but the practice is dismissed as ignorant and dangerous quackery by conventional medicine.

I suspect that there are acquired 'hormone resistance' or 'type II' versions of all the various endocrine disorders. These would produce the symptoms without reducing the levels of the hormones in the blood. However hormone treatments should still work, simply by overwhelming the resistance.

We know that diabetes comes in two forms, (type I) gland failure and (type II), 'insulin resistance', and that the resistance version is usually acquired rather than inborn. The mechanism for the resistance version of diabetes is mysterious.

There are known to be corresponding 'gland failure' and 'resistance' versions of diseases associated with all the other endocrine hormones, but for some reason the resistance versions are thought to be very rare, and only to be inherited, never acquired.

Should such acquired resistance mechanisms exist and be common, then on evolutionary grounds they would have to be caused by the direct action of pathogens, be a side effect of immune defense against such pathogens, or have an environmental cause. Nothing else would be stable.

Chronic Fatigue Syndrome often seems to start with an infection.

I thought until recently that the problem must be rather complex, and depend on subtle balances of hormones in a complicated system. The idea is so simple and obvious that if it were straightforwardly true, it isn't credible that it would have been missed.

But it turns out that there have been two formal studies of the simplest possible version of idea (treat the symptoms of hypothyroidism with thyroxine) in the medical literature. And they're all I've managed to find. Further examples would be most welcome.

The two studies are apparently contradictory, but there's no real contradiction, in fact the second supports the first.

The first:

Clinical Response to Thyroxine Sodium in Clinically Hypothyroid but Biochemically Euthyroid Patients
G. R. B. SKINNER MD DSc FRCPath FRCOG, D. HOLMES, A. AHMAD PhD, J. A. DAVIES BSc and J. BENITEZ MSc

was an open trial done in 2000, by Gordon Skinner in Birmingham.

Dr Skinner took 139 patients, all of whom had symptoms consistent with a clinical diagnosis of hypothyroidism.

Of these the majority had been diagnosed with CFS or ME or Post-Viral Fatigue Syndrome, but thirty had been diagnosed with Major Depression, which also has all the right symptoms.

Dr Skinner started off with small doses of thyroxine, and slowly increased the doses, to quite high levels, until the patients got better. He reported that they all got considerably better. In fact his results are phenomenally good.

He mentioned the possibility of placebo effect, and the necessity of ruling it by placebo-controlled blinded randomised trial in the paper, but thought it unlikely. Many of these patients had been seriously ill for many years, and had usually tried a lot of things already.

[ From the study ]  In the absence of a control group, a placebo effect cannot be excluded in this or any study. However, the average duration of illness was 7.5 years in patients who had usually undergone an alarming array of traditional and alternative medications without significant improvement as evidenced by their wish to seek further medical advice. Secondly, certain clinical features allowed objective assessment, namely change in appearance, hair or skin texture, reduction in size of tongue and thyroid gland and increase in pulse rate.

If these patients hadn't had a hormone resistance, he would have done them very serious harm! He kept increasing the dose until it worked, and the highest dose he used was 300mg of thyroxine. That's more than the amount you'd usually use to completely replace the output of a removed thyroid gland. Given that all these people had normal hormone levels to start with, if the patient was not resisting the hormone, this should have caused a range of extremely unpleasant symptoms, including death.

He mentions no adverse effects whatsoever.

Dr Skinner wrote to the British Medical Journal suggesting that thryoxine should be tried in cases where the clinical symptoms of hypothyroidism were present but the blood tests were normal.

This prompted a small trial:

Thyroxine treatment in patients with symptoms of hypothyroidism but thyroid function tests within the reference range: randomised double blind placebo controlled crossover trial  

M Anne Pollock, Alison Sturrock, Karen Marshall, Kate M Davidson, Christopher J G Kelly, Alex D McMahon, E Hamish McLaren

This trial looks very well designed and established that:

(a) There was a huge placebo effect in the patients

(b) Thyroxine is very strongly disliked by the healthy controls (they could tell it from placebo and hated it)

(c) The patient group couldn't tell the difference between thyroxine and placebo (on average).

This result is very interesting of itself, and I make no criticism of the brave GPs who organised it in response to Skinner's letter, but unfortunately it has been taken as a refutation of Skinner's methods. Which it is not. In fact it supports him.

In fact there are two obvious relevant differences between what they did and what Skinner did:

(i) They used a fixed dose for everyone (100mg thyroxine / day) and made no attempt to tailor the dose to the patient.

I suspect that this would have made Skinner's treatment less effective, but it should still have worked.

(ii) They used very different criteria for selecting their patients.

Skinner had carefully done a 'clinical diagnosis' of hypothyroidism, using 16 symptoms, most of which were present in the majority of his patients.

The criteria for the formal trial were:

At least three of the following symptoms for six months: tiredness, lethargy, weight gain or inability to lose weight, intolerance to cold, hair loss, or dry skin or hair.

So a fat person with dry hair who didn't get enough sleep would have qualified as a patient.

This is utterly inadequate as a diagnosis of hypothyroidism! It is a famously difficult disease to diagnose!

Their patient group would have consisted mainly of people who didn't have the clinical symptoms of hypothyroidism. (EDIT: Obviously these people would have had symptoms of *something*, and thus probably been ill, but they are equally valid as symptoms of mild anaemia, or mild diabetes, which also seem to go undiagnosed a lot. The whole trick with hypothyroidism was to tell the difference between it and other similar diseases.)

If the type II version is rare or non-existent, then it would have included no real patients at all.

If the type II version is very common, then at least some of the patient group should have had the disease Skinner said he could cure.

What I think must have happened here is that the treatment produced great improvements in a few patients, and caused unpleasant symptoms in all the rest. This averaged out to 'can't tell the difference between placebo and treatment'. Remember that healthy people can!

I deduce that Skinner's treatment works pretty much as well as he thought it did, and that the disease he was curing is very common indeed.

Can anyone explain these two studies in any other way?

Conclusion

When combined with Sarah Myhill's paper showing that the principal cause of chronic fatigue is 'mitochondrial dysfunction', and that the action of the thyroid hormone is to stimulate the mitochondria, I think the case for a 'thyroid hormone resistance' disease manifesting as Chronic Fatigue Syndrome is unanswerable.

At the very least, this should be investigated.

I now believe my own argument, which until I saw Skinner's paper appeared even to me to be a wild idea made up from shreds of mathematical intuition and questionable evidence from biased sources. I think that Skinner's treatment is unlikely to be optimal, and research into what is actually going on needs to be done.

The problem, if it does exist, is likely to be extremely widespread, and explain far more than the mystery of Chronic Fatigue Syndrome and Fibromyalgia. I immediately claim Major Depressive Disorder and Irritable Bowel Syndrome as alternative labels for: 'type II hypothyroidism'. There is a large cluster of these diseases, all mysterious, all with very similar symptoms, known as the 'central sensitivity syndromes'.

And I should like to add that 'blood cholesterol' was once a test for hypothyroidism, so there are probably implications for heart disease as well. Anyone interested in the wider implications might want to take a look at Broda Barnes' work. I started off thinking he was a lunatic. I'm now fairly sure he must have been right all along.

I think it's now urgent to bring this to the attention of the medical profession and the sufferers' groups. Has anyone got any ideas how to do that?

I'll admit it: I am confused about genetics and heritability. Not about the results of the various twin studies - Scott summarises them as "~50% of the variation is heritable and ~50% is due to non-shared environment", which seems generally correct.

But I am confused about what this means in practice, due to arguments like "contacts are very important for business success, rich people get much more contacts than poor people, yet business success is strongly correlated with genetic parent wealth" and such. Assuming that genetics strongly determines... most stuff... goes against so many things we know or think we know about how the world works. And by "we" I mean lots of different people with lots of different political views - genetic determinism means, for instance, that current variations in regulation and taxes are pretty unimportant for individual outcomes.

Now, there are many caveats about the genetic results, particularly that they measure the variance of a factor rather than its absolute importance (and hence you get results like variation in nutrition being almost invisible as an explanation for variation in height), but it's still hard to figure out what this all means.

Then we have Scott's latest post, which points out that "non-shared environment" is not the same as "nurture", since it includes, for instance, dumb luck.

However, "heritable" is not the same as as "nature", either. For instance, sexism and racial prejudices, if they are widespread, come under the "heritable" effects rather than the "environment" ones. And then it gets even more confusing.

Widespread prejudice is not "environment". Rarer prejudice is.

For instance, imagine that we lived in a very sexist society where women were not allowed to work at all. Then there would be an extremely high, almost perfect, correlation between "having a Y chromosome" and "having a job". But this would obviously be susceptible to a cultural fix.

Obviously racial effects can have the same effect. It covers anything visible. So a high heritability is compatible with genetics being a cause of competence, and/or prejudice against visible genetic characteristics being important ("Our results indicate that we either live in a meritocracy or a hive of prejudice!").

Note that as prejudices get less widespread, they move from showing up on the genetic variation, to showing up in the environmental variation side. So widespread prejudices create a "nature" effect, rarer ones create a "nurture" effect. Evenly reducing the magnitude of a prejudice, however, doesn't change the side it will show up on.

Positional genetic goods: Beauty... and IQ?

Let's zoom in on one of those visible genetic characteristics: beauty. As Robin Hanson is fond of pointing out, beautiful people are more successful, and are judged as more competent and cooperative than they actually are. Therefore if we have a gene that increases both beauty and IQ, we would expect it's impact on success to be high. In the presence of such a gene, the correlation between IQ and success would be higher than it should objectively be. This suggest a (small) note of caution on the "mutation load" hypotheses; if reducing mutation load increases factors such as beauty, then we would expect increased success without necessarily increased competence.

But is it possible that IQ itself is in part a positional good? Consider that success doesn't just depend on competence, but on social skills, ability to present yourself well in an interview, and how managers and peers judge you. If IQ affects or covaries with one or another of those skills, then we would be overemphasising the importance of IQ in competence. Thus attempts to genetically boost IQ could give less impact than expected. The person whose genome was changed would benefit, but at the (partial) expense of everyone else.

Do people know of experiments (or planned experiments) that disentangle these issues?

UPDATE 3/16/16: I decided to go to public school, because I was tired of all the little annoying stuff at my current school--especially the entitled kids and the entitled attitude in general. Everybody acts like they deserve something. It's very irritating.

The other reason I came to that decision was "exploration value". By moving to a new situation I learn whether I really am better off in the kind of environment offered by the public high school; even if it ends up being worse for me, at least I know what to avoid. If it's good, I know it's good; and if it makes no difference, I know that, too.

EDIT: Just to clarify, I attended public middle school with the same group of kids, so I won't have to worry too much about getting to know new people. I am still in occasional contact with my old friends. I talked to one of them for several hours just yesterday.

(I'm new here, though I've lurked, so if I break any rules or otherwise do something detrimental, please let me know and I will try to correct my mistake)

I currently go to a rather nice independent high school. I'm on significant financial aid, so I can afford it. The academics there are outstanding. However, being a boarding school (I go as a day student) it requests a lot of our free time. We are required to participate in adult-sanctioned activities at least six or seven hours a week, in addition to normal classes. This means that 1. I get home from school around 6pm and 2. (more importantly) it's very hard to socialize when you don't board at the school, and there's really very little besides drama or sports (neither of which I like very much) that people do after school and actually enjoy/make friends in.

I'm strongly considering transferring to my public school, which is unusually good for a public school, as a junior next year. The academics are not as great (classes are less discussion-based and there are not as many APs offered) but there is a strong amount of participation in stuff I might actually enjoy after school (math team, etc.) because we're not required to do anything after school. I've noticed that when people tell me I have to do things, I enjoy them much, much less. Also, I won't have a commute, which would be nice.

Everything else about the schools is more or less comparable.

I'm sure when I think about this decision I am biased in some way. I'm probably succumbing to the sunk cost fallacy (sometimes I think if I'd been at the public school the last two years, I'd rather be at the private school) or something like that. If not, I'm facing the problem of Buridan's Ass.

My question to you: should I transfer or not? I have thought very hard and consulted several intelligent people, and have not been able to come to any sort of conclusion.

Here is a new paper of mine (12 pages) on suspicious agreement between belief and values. The idea is that if your empirical beliefs systematically support your values, then that is evidence that you arrived at those beliefs through a biased belief-forming process. This is especially so if those beliefs concern propositions which aren’t probabilistically correlated with each other, I argue.

I have previously written several LW posts on these kinds of arguments (here and here; see also mine and ClearerThinking’s political bias test) but here the analysis is more thorough. See also Thrasymachus' recent post on the same theme.

LINK

Yoshua Bengio, one the world's leading expert on machine learning, and neural networks in particular, explains his view on these issues in an interview. Relevant quotes:

There are people who are grossly overestimating the progress that has been made. There are many, many years of small progress behind a lot of these things, including mundane things like more data and computer power. The hype isn’t about whether the stuff we’re doing is useful or not—it is. But people underestimate how much more science needs to be done. And it’s difficult to separate the hype from the reality because we are seeing these great things and also, to the naked eye, they look magical

[ Recursive self-improvement ] It’s not how AI is built these days. Machine learning means you have a painstaking, slow process of acquiring information through millions of examples. A machine improves itself, yes, but very, very slowly, and in very specialized ways. And the kind of algorithms we play with are not at all like little virus things that are self-programming. That’s not what we’re doing.

Right now, the way we’re teaching machines to be intelligent is that we have to tell the computer what is an image, even at the pixel level. For autonomous driving, humans label huge numbers of images of cars to show which parts are pedestrians or roads. It’s not at all how humans learn, and it’s not how animals learn. We’re missing something big. This is one of the main things we’re doing in my lab, but there are no short-term applications—it’s probably not going to be useful to build a product tomorrow.

We ought to be talking about these things [ AI risks ]. The thing I’m more worried about, in a foreseeable future, is not computers taking over the world. I’m more worried about misuse of AI. Things like bad military uses, manipulating people through really smart advertising; also, the social impact, like many people losing their jobs. Society needs to get together and come up with a collective response, and not leave it to the law of the jungle to sort things out.

I think it's fair to say that Bengio has joined the ranks of AI researchers like his colleagues Andrew Ng and Yann LeCun who publicly express skepticism towards imminent human-extinction-level AI.

Too many people attempt to use logic when they should be using probabilities - in fact, when they are using probabilities, but don't mention it. Here are some of the major fallacies caused by misusing logic and probabilities this way:

1. "It's not certain" does not mean "It's impossible" (and vice versa).
2. "We don't know" absolutely does not imply "It's impossible".
3. "There is evidence against it" doesn't mean much on its own.
4. Being impossible *in a certain model*, does not mean being impossible: it changes the issue to the probability of the model.

Common fallacies in probability

A few notes about the site mechanics

Excellent list by Brenton Mayer and Peter McIntyre: http://mcntyr.com/52-concepts-cognitive-toolkit/

I think the list can also serve as a useful index and/or introduction to a lot of LessWrong concepts.

A note of caution: I find that brief lists like this can actually be counterproductive, since they make you feel like you understand the issues when all you did was read a short definition and peg a name on the concept. I'd recommend doing the following: Look through the list carefully and slowly. If there's a concept there that you've read a lot about then you can go on to the next one, although do take a look at where the link points to in case it's to an interesting article you haven't seen before. If you haven't read a lot about the concept then ideally you should click on the link and read all about it. If you're more pressed for time, then at least take a few moments to reflect on each concept and think how it might apply to you. If you have even a slight suspicion that there might be something in the concept that wasn't completely obvious to you before, then click on the link even though you're pressed for time. If you're so time constrained that you can't even do this, then consider just bookmarking the list and getting back to it later. Personally I think it's better to read it later carefully than to read it now and think you understand it when you really don't.

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part Q: Joy in the Merely Real (pp. 939-979). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

Q. Joy in the Merely Real

202. Joy in the Merely Real - If you can't take joy in things that turn out to be explicable, you're going to set yourself up for eternal disappointment. Don't worry if quantum physics turns out to be normal.

203. Joy in Discovery - It feels incredibly good to discover the answer to a problem that nobody else has answered. And we should enjoy finding answers. But we really shouldn't base our joy on the fact that nobody else has done it before. Even if someone else knows the answer to a puzzle, if you don't know it, it's still a mystery to you. And you should still feel joy when you discover the answer.

204. Bind Yourself to Reality - There are several reasons why it's worth talking about joy in the merely real in a discussion on reductionism. One is to leave a line of retreat. Another is to improve your own abilities as a rationalist by learning to invest your energy in the real world, and in accomplishing things here, rather than in a fantasy.

205. If You Demand Magic, Magic Won't Help - Magic (and dragons, and UFOs, and ...) get much of their charm from the fact that they don't actually exist. If dragons did exist, people would treat them like zebras; most people wouldn't bother to pay attention, but some scientists would get oddly excited about them. If we ever create dragons, or find aliens, we will have to learn to enjoy them, even though they happen to exist.

206. Mundane Magic - A list of abilities that would be amazing if they were magic, or if only a few people had them.

207. The Beauty of Settled Science - Most of the stuff reported in Science News is false, or at the very least, misleading. Scientific controversies are topics of such incredible difficulty that even people in the field aren't sure what's true. Read elementary textbooks. Study the settled science before you try to understand the outer fringes.

208. Amazing Breakthrough Day: April 1st - A proposal for a new holiday, in which journalists report on great scientific discoveries of the past as if they had just happened, and were still shocking.

209. Is Humanism a Religion Substitute? - Trying to replace religion with humanism, atheism, or transhumanism doesn't work. If you try to write a hymn to the nonexistence of god, it will fail, because you are simply trying to imitate something that we don't really need to imitate. But that doesn't mean that the feeling of transcendence is something we should always avoid. After all, in a world in which religion never existed, people would still feel that same way.

210. Scarcity - Describes a few pieces of experimental evidence showing that objects or information which are believed to be in short supply are valued more than the same objects or information would be on their own.

211. The Sacred Mundane - There are a lot of bad habits of thought that have developed to defend religious and spiritual experience. They aren't worth saving, even if we discard the original lie. Let's just admit that we were wrong, and enjoy the universe that's actually here.

212. To Spread Science, Keep It Secret - People don't study science, in part, because they perceive it to be public knowledge. In fact, it's not; you have to study a lot before you actually understand it. But because science is thought to be freely available, people ignore it in favor of cults that conceal their secrets, even if those secrets are wrong. In fact, it might be better if scientific knowledge was hidden from anyone who didn't undergo the initiation ritual, and study as an acolyte, and wear robes, and chant, and...

213. Initiation Ceremony - Brennan is inducted into the Conspiracy.

This has been a collection of notes on the assigned sequence for this fortnight. The most important part of the reading group though is discussion, which is in the comments section. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

The next reading will cover Part R: Physicalism 201 (pp. 983-1078). The discussion will go live on Wednesday, 13 January 2016, right here on the discussion forum of LessWrong.

I feel I've learned a lot about problem solving from less wrong (and HPMOR in particular). be concrete, hold off on proposing solutions, et cetera. The effect*, unfortunately, doesn't seem to be as much as I had hoped.

Small increases in intelligence are extremely general and even recursive, so it feels worth the effort. But, I found alternatives much more effective (though still modest) then studying/discussing/applying the sequences. like meditation or smart drugs. 

I'm interested in other lesswrongers experiences in cognitive enhancement. 

* by "smarter" I mean "better at problem solving", where examples of "problems" are writing a program, finding the right thing to say to resolve interpersonal conflict, memorizing some random fact quickly and then recalling it quickly/vividly. let me know if you want further clarification. 

I am curious on the perspective of a rationalist discussion board (this seems like a good start) on the practice of magic. I introduce the novel, genius concept of "rationalist magic", i.e. magic practiced by rationalists.

Why would rationalists practice magic? That makes no sense!!

It's the logical conclusion to Making Peace with Belief, Engineering Religion and the self-help threads.

What would that look like?

Good question. Here are some possible considerations to make:

• It's given low probability that magic is more than purely mental phenomena. The practice is called "placebomancy" to make it clear that such an explanation is favoured.
• It is practiced as a way to gain placebons.
• A cult of rationalist magic, the Cult of Rationatron, should be formed to compete against worse (anti-rationality, anti-science, violent) cults.
• Rationalist groups can retain more members due to abundance of placebons.
• The ultimate goal is to use rationality to devise a logically optimal system of magic with which to build the Philosopher's Stone and fix the world like in HPMOR. (Just kidding, magic isn't real.)

I looked into magical literature and compiled a few placebo techniques/exercises, along with their informal instructions. These might be used as a starting point. If there are any scientific errors these can eventually be corrected. I favoured techniques that can be done with little to no preparation and provide some results. Of course, professional assistance (e.g. yoga classes) can also be helpful.

1. Mindfulness Meditation

• (Optional) Do 1-3 deep mouth-exhales to relax.
• Find a good position.
• Begin by being aware of breath.
• (Optional) Move on to calmly observing different parts of the body, vision, the other senses, thoughts, mandala visualization, and so on.
• (Optional) Compare the experience to teachings of buddhism.
• (Optional) Say "bud-" for each inhale and "-dho" for each exhale; alternatively, count them from 1 to 10 and reset each time.

Note that trying to focus on not focusing isn't always helpful. Hence the common technique is that of focusing on a single thing, or rather, simply being passively aware of it. The goal is to discipline the mind and develop one-pointedness.

2. Astral Projection (OBE)

• Lay on a bed, relax.
• Try out some of the tips from the previous exercise.
• Stay there for 30-60 minutes until you reach the hypnagogic state (a state where the mind is awake but the body is sleeping) and try to (a) feel your astral body and grab a rope, (b) feel vibrations in your body, (c) roll out of the body, (d) (...).

Astral Projection can be thought of as a very vivid stage of dreaming. Some authors have more detailed exercises related to this[7]. It might take many tries to do this exercise right.

3. Mantras

You can borrow an eastern mantra such as "om mani padme hum", "om namah shivaya" and "hare kṛiṣhṇa hare kṛiṣhṇa / kṛiṣhṇa kṛiṣhṇa hare hare / hare rāma hare rāma / rāma rāma hare hare" or make up some phrase. Whatever works for you.

Chanting mantras is a form of sensory excitation. Both sensory excitation and deprivation can induce trance. This can be used along with exercise 1.

(Optional) Find one of these.

4. Contemplation

• Take a moment to contemplate the harmony of the universe and/or have love towards some/all beings.

5. Idol Worship

• Make a shrine dedicated to Rationatron, god of rationality.

This and this are possible forms of Rationatron.

6. Minimal Spellcasting

• Make a wish.
• Clear your mind.
• Take a deep, long breath imagining that as you exhale the wish is being registered into the universe by Rationatron.

• Forget about the wish.

Magicians claim it's more magically effective to forget about the wish after casting the spell (fourth step) and let your subconscious act than to use the repeat-your-wish-every-day method.

This is, as far as I know, the simplest spellcasting technique. It can be complicated further by the addition of rituals, sigils, poses[6], and stronger methods of inducing trance in the second step.

7. Deity Generator

• Take any arbitrary concept or amalgam of concepts.
• (Optional) Associate it to a colour.
• (Optional) Make it a new deity.

For one reason or another, spiritualists love doing this.

This exercise can make arbitrary "powers" or "deities" for use in other exercises. For example, the association "yellow - wealth" is a "power" for exercise 6, or you might imagine yourself as a "deity" in that same exercise to induce some emotion.

8. Tulpa Making

This is a technique found in Tibetan buddhism lore[5] as one ability held by bodhisattvas and used by Buddha to multiply himself. This was adopted by communities of westerners in the internet who generally don't attribute mystical properties to the practice and made detailed tutorials (tulpa.info).

The technique uses your subconscious to create a companion. It consists in visualizing and talking to a being in your imagination until it eventually produces unexpected thoughts.

You might be asking, "can I model this companion after a cartoon?" The answer is yes.

Note: some of the following techniques might require further examination.

9. Aura Sight

Some authors[1,2,4] give exercises attributed to peripheral vision or meditation. If anyone finds out how to see auras, please confirm.

10. Invoking and Banishing Ritual of Truth

• Imagine a circle of protection surrounding you.
• (Invoking) "I open/invoke the powers of p, q, ¬p, ¬q."
• (Banishing) "I close/revoke/banish the powers of ¬q, ¬p, q, p."
• (Optional) This can be performed solely in the imagination through visualization or with more realism added to different degrees inbetween (e.g. by making a real circle, pointing a sword or your hand to the four directions).

This has been used for different purposes; as an introduction to ritual work or simply as routine.

The common formulation of this exercise uses a pentagram, holy names, the elements, planets and a bunch of other nonsense. Why do I have to remember all this roleplaying? Do they think this is D&D? Therefore, I designed a more efficient version of the technique that also replaces the magical symbolism with superior logic symbolism.

Note: these are roughly analogous[3] to the simpler placebo techniques known as shielding (imagining a shield), centering (regaining focus by being aware of the solar plexus/heart area) and grounding (putting your feet on the ground to receive/release energy from/to the ground).

11. Demonic Mirror Summoning

Call upon Dark Lord Voldemort and say the "avada kedavra" mantra 7+ times in front of a mirror in a dimly lit room until a demon pops up and/or your appearance gets distorted.

Some individuals report holding a conversation with their mirror self through mirror exercises.

FAQ

What is the purpose of this?

It's about time someone made an atheist religion.

Why not follow the Flying Spaggheti Monster religion instead?

It doesn't provide placebo techniques. It only functions as a point in argumentation.

Do I have to do all of the exercises?

No, only those that you personally deem helpful. However, the first exercise (meditation) is generally recommended by health research. It's also a pre-requisite to many other exercises. Note: although meditation is generally recommended, some caution, common sense and preparation is advised (specially for exercises 2-3).

What are the teachings?

It's acknowledged that rational people can sometimes get to different conclusions. Therefore, there is no mandatory teachings. However, it uses "rationality" as a starting point to distinguish it from other cults meaning that "placebo" is used as the default model of magic and that both logic and the use of such techniques is encouraged. It can be used as a gathering of placebo techniques for atheists and as a blank slate from the dogma of already existing cults on the nature of magic.

What is the pantheon of this religion?

The "official" pantheon is that of the universe itself (Einstein's pantheism; it's used in exercises 4 and 6), Rationatron (a deity of rationality) and Dark Lord Voldemort (the opposer). They fulfill different god-roles. More gods can be created with the Deity Generator exercise or borrowed.

Can I worship Eris, Cthullu or Horus/Isis/Odin...?

Yes, see above answer.

Wait, Dark Lord Voldemort? Really?

Christianity had lazier ways to come up with their demons and nobody noticed. Zing.

Aren't some of those techniques irrational?

Only when used by superstitious people. Once used by rationalists, they become super-rational.

What about black magic? Can I cast hexes?

They aren't going to work because magic is not real.

References

1. frater, ud. high magick. A good overview on different kinds of magic.

2. hine, phil. spirit guides. Another overview.

3. hine, phil. modern shamanism pt1-2. Overview for shamans.

4. samuel sagan. awakening the third eye.

5. alexandra david-neel. magic and mystery in tibet. A book on buddhist lore.

6. crowley. liber o.

7. robert bruce. mastering astral projection.

There is a fairly large contingent of safety research oriented people at NIPS this year. I'm unfortunately not among them, but if you're there and interested in connecting with others on AI safety topics or other LW issues - general rationality, EA, etc. I welcome you to make this thread a Schelling point to create meeting opportunities :). You can also PM me I can connect you to people I know are there.

Disclaimer: This post is mainly of interest to EA-oriented Less Wrongers

Happy to share that I got this article promoting effective giving, and especially advocating Against Malaria Foundation, GiveWell, and The Life You Can Save, published in The Huffington Post.

This piece is part of my broader work at Intentional Insights, a nonprofit devoted to promoting effective altruism and rational thinking to a broad audience effectively, by using modern promotion and marketing methods. Our goals with this and similar articles is to channel both money to effective charities and encourage people to think about donations in a rational, science-based, data-driven manner. These articles are also aimed to be a good fit for those supportive of EA ideas to share with others on social media, to help encourage non-EAs to adopt effective giving strategies, since the articles are aimed to be easy to read and engaging.

For a while now I've been trying hard to understand philosophical viewpoints that defer from mine. Somewhere along the line I've picked up or developed a lot of the LW-typical viewpoints (not sure if this was because of LW, or if I developed them earlier and that's what later attracted me to LW), but I know there are a lot of smart people out there who disagree with those viewpoints. I've tried to read articles and books on this, but they either don't address what I'm looking for somehow, or they're so technical that I have a hard time following them. I've also talked at some length with a philosophy professor, but our conversations often seem to end with me still being confused and the professor being confused about what it is I might be confused about.

I'm thinking maybe it'll help to get some input from people who do intuitively agree with my viewpoints, hence this post. So, can someone please tell me what the central arguments or motivations are for promoting the following:

Epistemology:

• Trusting philosophical intuitions and/or the way people use words to the point of making strong metaphysical claims about the world, despite the findings of cognitive science / evolutionary psychology / experimental philosophy / etc. that there doesn't seem to be any good reason to trust those intuitions / ways of talking
• Not looking at the world in a probabilistic way
• Using personal preference or personal intuitions as priors instead of some objective measure along the lines of Solomonoff Induction

Ontology / philosophy of mind:

• Moral realism
• Mathematical Platonism
• Libertarian free will (I'm looking for arguments other than those from religion)
• The view that there actually exist abstract "tables" and "chairs" and not just particles arranged into those forms
• The existence of non-physical minds (I'm looking for arguments other than the argument from the Hard Problem of Consciousness)

I suspect I'm having trouble with the ontology issues because of my trouble understanding the epistemology issues. Specifically, I keep getting the impression that most (all?) of the arguments for the ontology issues boil down to trusting philosophical intuitions and/or the way people use words. Something along the following lines:

I intuitively feel that there really are objective morals (or: objective mathematics, actual free will, tables and chairs, minds).

Therefore, there really are objective morals (etc.).

Or the equivalent using the way people talk about things.

But this just seems totally ludicrous to me. If we trust cognitive science, evolutionary psychology, etc., and if those fields give us perfectly plausible reasons for why we might intuitively feel this way / talk this way, even if it didn't reflect the truth, then what could possibly be your motivation for sticking to your intuitions anyway and using them to support some grand metaphysical theory?

The only thing I can think of is that people who support using intuitions like this say, "well, you're also ultimately basing yourself on intuitions for things like logic, existence of mind-independent objects, Occamian priors, and all the other viewpoints that you view as intuitively plausible, so I can jolly well use whatever intuitions I feel like too." But although I can hear such words and why they sound reasonable in a sense, they still seem totally crazy to me, although I'm not 100% sure why.

Any help would be appreciated.

Richard Kennaway has posted about an edit war on the wiki. Richard, thank you.

Unfortunately, I've only used the wiki a little, and don't have a feeling for why the edit history for an article is inaccessible. Is the wiki broken or has someone found a way to hack it? Let it be known that hacking the wiki is something I'll ban for.

VoiceofRa, I'd like to know why you deleted Gleb's article. Presumably you have some reason for why you think it was unsatisfactory.

I'm also notifying tech in the hope of finding out what happened to the edit history.

I'm working on a conlang (constructed language) and would like some input from the Less Wrong community.  One of the goals is to investigate the old Sapir-Whorf hypothesis regarding language affecting cognition.  Does anyone here have any ideas regarding linguistic mechanisms that would encourage more rational thinking, apart from those that are present in the oft-discussed conlangs e-prime, loglan, and its offshoot lojban?  Or perhaps mechanisms that are used in one of those conlangs, but might be buried too deeply for a person such as myself, who only has superficial knowledge about them, to have recognized?  Any input is welcomed, from other conlangs to crazy ideas.

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part N: A Human's Guide to Words (pp. 677-801) and Interlude: An Intuitive Explanation of Bayes's Theorem (pp. 803-826). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

N. A Human's Guide to Words

153. The Parable of the Dagger - A word fails to connect to reality in the first place. Is Socrates a framster? Yes or no?

154. The Parable of Hemlock - Your argument, if it worked, could coerce reality to go a different way by choosing a different word definition. Socrates is a human, and humans, by definition, are mortal. So if you defined humans to not be mortal, would Socrates live forever?

You try to establish any sort of empirical proposition as being true "by definition". Socrates is a human, and humans, by definition, are mortal. So is it a logical truth if we empirically predict that Socrates should keel over if he drinks hemlock? It seems like there are logically possible, non-self-contradictory worlds where Socrates doesn't keel over - where he's immune to hemlock by a quirk of biochemistry, say. Logical truths are true in all possible worlds, and so never tell you which possible world you live in - and anything you can establish "by definition" is a logical truth.

You unconsciously slap the conventional label on something, without actually using the verbal definition you just gave. You know perfectly well that Bob is "human", even though, on your definition, you can never call Bob "human" without first observing him to be mortal.

155. Words as Hidden Inferences - The mere presence of words can influence thinking, sometimes misleading it.

The act of labeling something with a word, disguises a challengable inductive inference you are making. If the last 11 egg-shaped objects drawn have been blue, and the last 8 cubes drawn have been red, it is a matter of induction to say this rule will hold in the future. But if you call the blue eggs "bleggs" and the red cubes "rubes", you may reach into the barrel, feel an egg shape, and think "Oh, a blegg."

156. Extensions and Intensions - You try to define a word using words, in turn defined with ever-more-abstract words, without being able to point to an example. "What is red?" "Red is a color." "What's a color?" "It's a property of a thing?" "What's a thing? What's a property?" It never occurs to you to point to a stop sign and an apple.

The extension doesn't match the intension. We aren't consciously aware of our identification of a red light in the sky as "Mars", which will probably happen regardless of your attempt to define "Mars" as "The God of War".

157. Similarity Clusters - Your verbal definition doesn't capture more than a tiny fraction of the category's shared characteristics, but you try to reason as if it does. When the philosophers of Plato's Academy claimed that the best definition of a human was a "featherless biped", Diogenes the Cynic is said to have exhibited a plucked chicken and declared "Here is Plato's Man." The Platonists promptly changed their definition to "a featherless biped with broad nails".

158. Typicality and Asymmetrical Similarity - You try to treat category membership as all-or-nothing, ignoring the existence of more and less typical subclusters. Ducks and penguins are less typical birds than robins and pigeons. Interestingly, a between-groups experiment showed that subjects thought a disease was more likely to spread from robins to ducks on an island, than from ducks to robins.

159. The Cluster Structure of Thingspace - A verbal definition works well enough in practice to point out the intended cluster of similar things, but you nitpick exceptions. Not every human has ten fingers, or wears clothes, or uses language; but if you look for an empirical cluster of things which share these characteristics, you'll get enough information that the occasional nine-fingered human won't fool you.

160. Disguised Queries - You ask whether something "is" or "is not" a category member but can't name the question you really want answered. What is a "man"? Is Barney the Baby Boy a "man"? The "correct" answer may depend considerably on whether the query you really want answered is "Would hemlock be a good thing to feed Barney?" or "Will Barney make a good husband?"

161. Neural Categories - You treat intuitively perceived hierarchical categories like the only correct way to parse the world, without realizing that other forms of statistical inference are possible even though your brain doesn't use them. It's much easier for a human to notice whether an object is a "blegg" or "rube"; than for a human to notice that red objects never glow in the dark, but red furred objects have all the other characteristics of bleggs. Other statistical algorithms work differently.

162. How An Algorithm Feels From Inside - You talk about categories as if they are manna fallen from the Platonic Realm, rather than inferences implemented in a real brain. The ancient philosophers said "Socrates is a man", not, "My brain perceptually classifies Socrates as a match against the 'human' concept".

You argue about a category membership even after screening off all questions that could possibly depend on a category-based inference. After you observe that an object is blue, egg-shaped, furred, flexible, opaque, luminescent, and palladium-containing, what's left to ask by arguing, "Is it a blegg?" But if your brain's categorizing neural network contains a (metaphorical) central unit corresponding to the inference of blegg-ness, it may still feel like there's a leftover question.

163. Disputing Definitions - You allow an argument to slide into being about definitions, even though it isn't what you originally wanted to argue about. If, before a dispute started about whether a tree falling in a deserted forest makes a "sound", you asked the two soon-to-be arguers whether they thought a "sound" should be defined as "acoustic vibrations" or "auditory experiences", they'd probably tell you to flip a coin. Only after the argument starts does the definition of a word become politically charged.

164. Feel the Meaning - You think a word has a meaning, as a property of the word itself; rather than there being a label that your brain associates to a particular concept. When someone shouts, "Yikes! A tiger!", evolution would not favor an organism that thinks, "Hm... I have just heard the syllables 'Tie' and 'Grr' which my fellow tribemembers associate with their internal analogues of my owntiger concept and which aiiieeee CRUNCH CRUNCH GULP." So the brain takes a shortcut, and it seems that the meaning of tigerness is a property of the label itself. People argue about the correct meaning of a label like "sound".

165. The Argument from Common Usage - You argue over the meanings of a word, even after all sides understand perfectly well what the other sides are trying to say. The human ability to associate labels to concepts is a tool for communication. When people want to communicate, we're hard to stop; if we have no common language, we'll draw pictures in sand. When you each understand what is in the other's mind, you are done.

You pull out a dictionary in the middle of an empirical or moral argument. Dictionary editors are historians of usage, not legislators of language. If the common definition contains a problem - if "Mars" is defined as the God of War, or a "dolphin" is defined as a kind of fish, or "Negroes" are defined as a separate category from humans, the dictionary will reflect the standard mistake.

You pull out a dictionary in the middle of any argument ever. Seriously, what the heck makes you think that dictionary editors are an authority on whether "atheism" is a "religion" or whatever? If you have any substantive issue whatsoever at stake, do you really think dictionary editors have access to ultimate wisdom that settles the argument?

You defy common usage without a reason, making it gratuitously hard for others to understand you. Fast stand up plutonium, with bagels without handle.

166. Empty Labels - You use complex renamings to create the illusion of inference. Is a "human" defined as a "mortal featherless biped"? Then write: "All [mortal featherless bipeds] are mortal; Socrates is a [mortal featherless biped]; therefore, Socrates is mortal." Looks less impressive that way, doesn't it?

167. Taboo Your Words - If Albert and Barry aren't allowed to use the word "sound", then Albert will have to say "A tree falling in a deserted forest generates acoustic vibrations", and Barry will say "A tree falling in a deserted forest generates no auditory experiences". When a word poses a problem, the simplest solution is to eliminate the word and its synonyms.

168. Replace the Symbol with the Substance - The existence of a neat little word prevents you from seeing the details of the thing you're trying to think about. What actually goes on in schools once you stop calling it "education"? What's a degree, once you stop calling it a "degree"? If a coin lands "heads", what's its radial orientation? What is "truth", if you can't say "accurate" or "correct" or "represent" or "reflect" or "semantic" or "believe" or "knowledge" or "map" or "real" or any other simple term?

169. Fallacies of Compression - You have only one word, but there are two or more different things-in-reality, so that all the facts about them get dumped into a single undifferentiated mental bucket. It's part of a detective's ordinary work to observe that Carol wore red last night, or that she has black hair; and it's part of a detective's ordinary work to wonder if maybe Carol dyes her hair. But it takes a subtler detective to wonder if there are two Carols, so that the Carol who wore red is not the same as the Carol who had black hair.

170. Categorizing Has Consequences - You see patterns where none exist, harvesting other characteristics from your definitions even when there is no similarity along that dimension. In Japan, it is thought that people of blood type A are earnest and creative, blood type Bs are wild and cheerful, blood type Os are agreeable and sociable, and blood type ABs are cool and controlled.

171. Sneaking in Connotations - You try to sneak in the connotations of a word, by arguing from a definition that doesn't include the connotations. A "wiggin" is defined in the dictionary as a person with green eyes and black hair. The word "wiggin" also carries the connotation of someone who commits crimes and launches cute baby squirrels, but that part isn't in the dictionary. So you point to someone and say: "Green eyes? Black hair? See, told you he's a wiggin! Watch, next he's going to steal the silverware."

172. Arguing "By Definition" - You claim "X, by definition, is a Y!" On such occasions you're almost certainly trying to sneak in a connotation of Y that wasn't in your given definition. You define "human" as a "featherless biped", and point to Socrates and say, "No feathers - two legs - he must be human!" But what you really care about is something else, like mortality. If what was in dispute was Socrates's number of legs, the other fellow would just reply, "Whaddaya mean, Socrates's got two legs? That's what we're arguing about in the first place!"

You claim "Ps, by definition, are Qs!" If you see Socrates out in the field with some biologists, gathering herbs that might confer resistance to hemlock, there's no point in arguing "Men, by definition, are mortal!" The main time you feel the need to tighten the vise by insisting that something is true "by definition" is when there's other information that calls the default inference into doubt.

You try to establish membership in an empirical cluster "by definition". You wouldn't feel the need to say, "Hinduism, by definition, is a religion!" because, well, of course Hinduism is a religion. It's not just a religion "by definition", it's, like, an actual religion. Atheism does not resemble the central members of the "religion" cluster, so if it wasn't for the fact that atheism is a religion by definition, you might go around thinking that atheism wasn't a religion. That's why you've got to crush all opposition by pointing out that "Atheism is a religion" is true by definition, because it isn't true any other way.

173. Where to Draw the Boundary? - Your definition draws a boundary around things that don't really belong together. You can claim, if you like, that you are defining the word "fish" to refer to salmon, guppies, sharks, dolphins, and trout, but not jellyfish or algae. You can claim, if you like, that this is merely a list, and there is no way a list can be "wrong". Or you can stop playing nitwit games and admit that you made a mistake and that dolphins don't belong on the fish list.

174. Entropy, and Short Codes - You use a short word for something that you won't need to describe often, or a long word for something you'll need to describe often. This can result in inefficient thinking, or even misapplications of Occam's Razor, if your mind thinks that short sentences sound "simpler". Which sounds more plausible, "God did a miracle" or "A supernatural universe-creating entity temporarily suspended the laws of physics"?

175. Mutual Information, and Density in Thingspace - You draw your boundary around a volume of space where there is no greater-than-usual density, meaning that the associated word does not correspond to any performable Bayesian inferences. Since green-eyed people are not more likely to have black hair, or vice versa, and they don't share any other characteristics in common, why have a word for "wiggin"?

176. Superexponential Conceptspace, and Simple Words - You draw an unsimple boundary without any reason to do so. The act of defining a word to refer to all humans, except black people, seems kind of suspicious. If you don't present reasons to draw that particular boundary, trying to create an "arbitrary" word in that location is like a detective saying: "Well, I haven't the slightest shred of support one way or the other for who could've murdered those orphans... but have we considered John Q. Wiffleheim as a suspect?"

177. Conditional Independence, and Naive Bayes - You use categorization to make inferences about properties that don't have the appropriate empirical structure, namely, conditional independence given knowledge of the class, to be well-approximated by Naive Bayes. No way am I trying to summarize this one. Just read the blog post.

178. Words as Mental Paintbrush Handles - You think that words are like tiny little LISP symbols in your mind, rather than words being labels that act as handles to direct complex mental paintbrushes that can paint detailed pictures in your sensory workspace. Visualize a "triangular lightbulb". What did you see?

179. Variable Question Fallacies - You use a word that has different meanings in different places as though it meant the same thing on each occasion, possibly creating the illusion of something protean and shifting."Martin told Bob the building was on his left." But "left" is a function-word that evaluates with a speaker-dependent variable grabbed from the surrounding context. Whose "left" is meant, Bob's or Martin's?

180. 37 Ways That Words Can Be Wrong - Contains summaries of the sequence of posts about the proper use of words.

Interlude: An Intuitive Explanation of Bayes's Theorem - Exactly what it says on the tin.

This has been a collection of notes on the assigned sequence for this fortnight. The most important part of the reading group though is discussion, which is in the comments section. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

The next reading will cover The World: An Introduction (pp. 834-839) and Part O: Lawful Truth (pp. 843-883). The discussion will go live on Wednesday, 2 December 2015, right here on the discussion forum of LessWrong.

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

Winter is coming, and so is Solstice season. There'll be large rationality-centric-or-adjaecent events in NYC, the Bay Area, and Seattle(and possibly other places - if you're interested in running a Solstice event or learning what that involves, send me a PM). In NYC, there'll be a general megameetup throughout the weekend, for people who want to stay through Sunday afternoon, and if you're interested in shared housing you can fill out this form.

The NYC Solstice isn't running a kickstarter this year, but I'll need to pay for the venue by November 19th ($6125). So if you are planning on coming it's helpful to purchase tickets sooner rather than later. (Or preorder the next album or 2016 Book of Traditions, if you can't attend but want to support the event).

-

This is the second post in the Solstice 2015 sequence, discussing plans and musings on the potential cultural impact of the Solstice. The first post was here. 

This explores the Solstice's relationship with Atheism, Rationality, and Death.

Atheism

Some may be surprised that I don't consider atheism particularly core to the Solstice.

This probably will remain a part of it for the forseeable future. Atheists happen to be the demographic most hungry for some kind of meaningful winter traditions. And Beyond the Reach of God, a powerful essay that (often) plays an important role in the holiday, happens to frame it's argument around the non-existence of God.

But this doesn't actually seem especially inevitable or necessary. Beyond the Reach of God *isn't* about God, per se (at least, I don't see it that way). It's about the absolute, unforgiving neutrality of the laws of physics. It's about all the other sacred things that even atheists believe in, which they may make excuses for.

I think it's *currently* useful for there to a moment where we acknowledge that there is no God to bail us out, and that this is really important. But this may not always be the case. I would be pretty happy if, in 50 years, all references to God were gone from the Solstice (because the question of God was no longer one that preoccupied our society in the first place), but those crucial points were made in other ways. It can be a holiday for atheists without being about that in any specific way.

Rationality

It's common throughout the secular world to speak highly of "rationality." But oftentimes, what that means in practice is pointing out the mistakes that other people are making, the fallacies they're committing.

The brand of rationality that spawned the Solstice has a different meaning: a specific dedication to looking at the way your own mind and beliefs are flawed, and actively seeking to correct them. Looking for the sacred cows of your culture (be it liberal, libertarian, academic or otherwise) and figuring out how they have blinded you.

Rationality is... sort of a central theme, but in an understated way. It underlies everything going on in the event, but hasn't really been a central character.

This might be a mistake. In particular because rationality's role is very subtle, and easy to be missed. Axial Tilt is the reason for the season, not crazy sun gods. But the reason that's important is a larger principle: that beliefs are entangled, that habits of excuse-making for outdated beliefs can be dangerous -- and that this can apply not just to antiquated beliefs about sun gods but (more importantly) to your current beliefs about politics and finance and love and relationships.

Aesthetically, in a culture of rationalists, I think it's correct for "rationality" to be very understated at the Solstice - there are plenty of other times to dwell upon it. But since Solstice is going to get promoted outside of the culture that spawned it, it's possible it may be best to include songs or stories that make it's epistemic core more explicit, so as not to be forgotten. It would be very easy for the Solstice to become about making fun of religion, and that is very much not my goal.
 
This year I have a story planned that will end up putting this front and center, but that won't make for a very good "permanent" feature of the Solstice. I'm interested in people's comments on how to address that in a more longterm way.

Death

I think one of the most valuable elements of the Solstice is the way it addresses' death. Atheists or "nones" don't really have a centralized funeral culture, and this can actually be a problem - it means that when someone dies, you suddenly have to scramble to put together an event that feels earnest and true, that helps you grapple with one of life's harshest events, and many people are too overwhelmed to figure out how to do so.

Funerals, more than any kind of secular ceremony, benefit from prior ritualization - a set of clear instructions on what to do that feel familiar and comfortable. It's the not the time to experiment with novel, crazy ideas, even genuinely good ones.

So Solstice provides a venue to test out pieces of funeral ritual, and let the good ones become familiar. It also provides a time, in the interim, for people who haven't had the chance to grieve properly because their loved one's funeral was theistic-by-default.

I think for this to work optimally, it needs to be a bit more deliberate. There's a lot of death-centric songs in the Solstice (probably more than there should be), but relatively few that actually feel appropriate for a funeral. I'd like to look for opportunities to do things more directly-funeral-relevant, while still appropriate for the overall Solstice arc.

There's also a deeper issue here: secular folk vary wildly in how they relate to death. Some people are looking for a way to accept it. Other people think the very idea of accepting death is appalling.

Common Ground

I have my own opinions here, and I'll dive a bit more deeply into this in my next post. But for now, I'll just note that I want to help shape a funeral culture that does feel distinctive, with traditions that feel at least a little oddly specific (to avoid a sort of generic, store-brand feel), but which also strike a kind of timeless, universal chord. Funerals are a time when wildly disparate friends and family need to come together and find common ground.

When my grandmother died, I went to a Catholic mass. Two hundred people spoke in unison "our father, who art in heaven, hallowed be thy name." The words themselves meant very little, but the fact that two hundred people who speak them flawlessly together felt very meaningful to me. And I imagine it'd have been even more meaningful, if I believed in them.

In the secular world, not everyone's into chanting things as a group. But it still seems to me that having words that are familiar to you, which you can at least listen together and know that two hundred people also find them meaningful, could be very important.

Now, humanity has certainly not lacked for beautiful poetry surrounding death. Nor even beautiful non-supernatural poetry surrounding death. Nor even beautiful poetry-surrounding-death-that-matches-you-(yes-you)-'re-specific-worldview-surrounding-death. But what it does seem to be lacking is are well-known cultural artifacts that a wide array of people would feel comforted by, in a very primal way.

There's a particular poem that's meaningful to me. There's another poem (very similar, both relating to the turning of the seasons and our changing relationship with the seasons of over time), that's meaningful to my girlfriend. But they're just different enough that neither would be feel safe and familiar to both of us, in the event of someone's death.

So something I'd like to do with the Solstice, is to coordinate (across all Solstices, across the nation, and perhaps in other holidays and events) to find words or activities to share, that can become well known enough that everyone at a funeral could feel united.

An actionable question:

In particular, I think I'm looking for a poem (not intended to be the only element-addressing-death in the Solstice, but one that has a shot at widespread adoption),  with a few qualities:

 - Short enough (or with a simple refrain) that people can speak it aloud together.
 - Whether metaphorical or not, hints at a theme of relating to memories and the preserving thereof. (I think this is something most worldviews can relate to)
 - All things being equal, something fairly commonly known.
 - Since everyone's going to want their own favorite poem to be the one adopted, people interested in this problem should try applying some meta-cooperative-considerations - what do you wish other people with their own favorite poems were doing to try and settle on this?

If you have either suggestions for a poetic contender, or disagreements with my thought process here, let me know!

-

In the next (probably final) post of this mini-sequence, I'll be talking about Humanism, Transhumanism, and the Far Future.

This is a bit rough, but I think that it is an interesting and potentially compelling idea. To keep this short, and accordingly increase the number of eyes over it, I have only sketched the bare bones of the idea. 

     1)      Empirically, people have varying intuitions and beliefs about causality, particularly in Newcomb-like problems (http://wiki.lesswrong.com/wiki/Newcomb's_problem, http://philpapers.org/surveys/results.pl, and https://en.wikipedia.org/wiki/Irresistible_grace).

     2)      Also, as an empirical matter, some people believe in taking actions after the fact, such as one-boxing, or Calvinist “irresistible grace”, to try to ensure or conform with a seemingly already determined outcome. This might be out of a sense of retrocausality, performance, moral honesty, etc. What matters is that we know that they will act it out, despite it violating common sense causality. There has been some great work on decision theory on LW about trying to thread this needle well.

     3)      The second disjunct of the simulation argument (http://wiki.lesswrong.com/wiki/Simulation_argument) shows that the decision making of humanity is evidentially relevant in what our subjective credence should be that we are in a simulation. That is to say, if we are actively headed toward making simulations, we should increase our credence of being in a simulation, if we are actively headed away from making simulations, through either existential risk or law/policy against it, we should decrease our credence.

      4)      Many, if not most, people would like for there to be a pleasant afterlife after death, especially if we could be reunited with loved ones.

     5)      There is no reason to believe that simulations which are otherwise nearly identical copies of our world, could not contain, after the simulated bodily death of the participants, an extremely long-duration, though finite, "heaven"-like afterlife shared by simulation participants.

     6)      Our heading towards creating such simulations, especially if they were capable of nesting simulations, should increase credence that we exist in such a simulation and should perhaps expect a heaven-like afterlife of long, though finite, duration.

     7)      Those who believe in alternative causality, or retrocausality, in Newcomb-like situations should be especially excited about the opportunity to push the world towards surviving, allowing these types of simulations, and creating them, as it would potentially suggest, analogously, that if they work towards creating simulations with heaven-like afterlives, that they might in some sense be “causing” such a heaven to exist for themselves, and even for friends and family who have already died. Such an idea of life-after-death, and especially for being reunited with loved ones, can be extremely compelling.

     8)      I believe that people matching the above description, that is, holding both an intuition in alternative causality, and finding such a heaven-like-afterlife compelling, exist. Further, the existence of such people, and their associated motivation to try to create such simulations, should increase the credence even of two-boxing types, that we already live in such a world with a heaven-like afterlife. This is because knowledge of a motivated minority desiring simulations should increase credence in the likely success of simulations. This is essentially showing that “this probably happened before, one level up” from the two-box perspective.

     9)      As an empirical matter, I also think that there are people who would find the idea of creating simulations with heaven-like afterlives compelling, even if they are not one-boxers, from a simply altruistic perspective, both since it is a nice thing to do for the future sim people, who can, for example, probabilistically have a much better existence than biological children on earth can, and as it is a nice thing to do to increase the credence (and emotional comfort) of both one-boxers and two-boxers in our world thinking that there might be a life after death.

     10)   This creates the opportunity for a secular movement in which people work towards creating these simulations, and use this work and potential success in order to derive comfort and meaning from their life. For example, making donations to a simulation-creating or promoting, or existential threat avoiding, think-tank after a loved one’s death, partially symbolically, partially hopefully.

     11)   There is at least some room for Pascalian considerations even for two-boxers who allow for some humility in their beliefs. Nozick believed one-boxers will become two boxers if Box A is raised to 900,000, and two-boxers will become one-boxers if Box A is lowered to $1. Similarly, trying to work towards these simulations, even if you do not find it altruistically compelling, and even if you think that the odds of alternative or retrocausality is infinitesimally small, might make sense in that the reward could be extremely large, including potentially trillions of lifetimes worth of time spent in an afterlife “heaven” with friends and family.

Finally, this idea might be one worth filling in (I have been, in my private notes for over a year, but am a bit shy to debut that all just yet, even working up the courage to post this was difficult) if only because it is interesting, and could be used as a hook to get more people interested in existential risk, including the AI control problem. This is because existential catastrophe is probably the best enemy of credence in the future of such simulations, and accordingly in our reasonable credence in thinking that we have such a heaven awaiting us after death now. A short hook headline like “avoiding existential risk is key to afterlife” can get a conversation going. I can imagine Salon, etc. taking another swipe at it, and in doing so, creating publicity which would help in finding more similar minded folks to get involved in the work of MIRI, FHI, CEA etc. There are also some really interesting ideas about acausal trade, and game theory between higher and lower worlds, as a form of “compulsion” in which they punish worlds for not creating heaven containing simulations (therefore effecting their credence as observers of the simulation), in order to reach an equilibrium in which simulations with heaven-like afterlives are universal, or nearly universal. More on that later if this is received well.

Also, if anyone would like to join with me in researching, bull sessioning, or writing about this stuff, please feel free to IM me. Also, if anyone has a really good, non-obvious pin with which to pop my balloon, preferably in a gentle way, it would be really appreciated. I am spending a lot of energy and time on this if it is fundamentally flawed in some way.

Thank you.

*******************************

November 11 Updates and Edits for Clarification

     1)      There seems to be confusion about what I mean by self-location and credence. A good way to think of this is the Sleeping Beauty Problem (https://wiki.lesswrong.com/wiki/Sleeping_Beauty_problem)

If I imagine myself as Sleeping Beauty (and who doesn’t?), and I am asked on Sunday what my credence is that the coin will be tails, I will say 1/2. If I am awakened during the experiment without being told which day it is and am asked what my credence is that the coin was tails, I will say 2/3. If I am then told it is Monday, I will update my credence to ½. If I am told it is Tuesday I update my credence to 1. If someone asks me two days after the experiment about my credence of it being tails, if I somehow do not know the days of the week still, I will say ½. Credence changes with where you are, and with what information you have. As we might be in a simulation, we are somewhere in the “experiment days” and information can help orient our credence. As humanity potentially has some say in whether or not we are in a simulation, information about how humans make decisions about these types of things can and should effect our credence.

Imagine Sleeping Beauty is a lesswrong reader. If Sleeping Beauty is unfamiliar with the simulation argument, and someone asks her about her credence of being in a simulation, she probably answers something like 0.0000000001% (all numbers for illustrative purposes only). If someone shows her the simulation argument, she increases to 1%. If she stumbles across this blog entry, she increases her credence to 2%, and adds some credence to the additional hypothesis that it may be a simulation with an afterlife. If she sees that a ton of people get really interested in this idea, and start raising funds to build simulations in the future and to lobby governments both for great AI safeguards and for regulation of future simulations, she raises her credence to 4%. If she lives through the AI superintelligence explosion and simulations are being built, but not yet turned on, her credence increases to 20%. If humanity turns them on, it increases to 50%. If there are trillions of them, she increases her credence to 60%. If 99% of simulations survive their own run-ins with artificial superintelligence and produce their own simulations, she increases her credence to 95%. 

2)  This set of simulations does not need to recreate the current world or any specific people in it. That is a different idea that is not necessary to this argument. As written the argument is premised on the idea of creating fully unique people. The point would be to increase our credence that we are functionally identical in type to the unique individuals in the simulation. This is done by creating ignorance or uncertainty in simulations, so that the majority of people similarly situated, in a world which may or may not be in a simulation, are in fact in a simulation. This should, in our ignorance, increase our credence that we are in a simulation. The point is about how we self-locate, as discussed in the original article by Bostrom. It is a short 12-page read, and if you have not read it yet, I would encourage it:  http://simulation-argument.com/simulation.html. The point about past loved ones I was making was to bring up the possibility that the simulations could be designed to transfer people to a separate after-life simulation where they could be reunited after dying in the first part of the simulation. This was not about trying to create something for us to upload ourselves into, along with attempted replicas of dead loved ones. This staying-in-one simulation through two phases, a short life, and relatively long afterlife, also has the advantage of circumventing the teletransportation paradox as “all of the person" can be moved into the afterlife part of the simulation.  

Three known attempts to make a map of x-risks prevention in the field of science exist:

1. First is the list from the Global Catastrophic Risks Institute in 2012-2013, and many links there are already not working:

2. The second was done by S. Armstrong in 2014

3. And the most beautiful and useful map was created by Andrew Critch. But its ecosystem ignores organizations which have a different view of the nature of global risks (that is, they share the value of x-risks prevention, but have another world view).

In my map I have tried to add all currently active organizations which share the value of global risks prevention.

It also regards some active independent people as organizations, if they have an important blog or field of research, but not all people are mentioned in the map. If you think that you (or someone) should be in it, please write to me at alexei.turchin@gmail.com

I used only open sources and public statements to learn about people and organizations, so I can’t provide information on the underlying net of relations.

I tried to give all organizations a short description based on its public statement and also my opinion about its activity. 

In general it seems that all small organizations are focused on their collaboration with larger ones, that is MIRI and FHI, and small organizations tend to ignore each other; this is easily explainable from the social singnaling theory. Another explanation is that larger organizations have a great ability to make contacts.

It also appears that there are several organizations with similar goal statements. 

It looks like the most cooperation exists in the field of AI safety, but most of the structure of this cooperation is not visible to the external viewer, in contrast to Wikipedia, where contributions of all individuals are visible. 

It seems that the community in general lacks three things: a united internet forum for public discussion, an x-risks wikipedia and an x-risks related scientific journal.

Ideally, a forum should be used to brainstorm ideas, a scientific journal to publish the best ideas, peer review them and present them to the outer scientific community, and a wiki to collect results.

Currently it seems more like each organization is interested in creating its own research and hoping that someone will read it. Each small organization seems to want to be the only one to present the solutions to global problems and gain full attention from the UN and governments. It raises the problem of noise and rivalry; and also raises the problem of possible incompatible solutions, especially in AI safety.

The pdf is here: http://immortality-roadmap.com/riskorg5.pdf

The University of Cambridge Centre for the Study of Existential Risk (CSER) is recruiting for an Academic Project Manager. This is an opportunity to play a shaping role as CSER builds on its first year's momentum towards becoming a permanent world-class research centre. We seek an ambitious candidate with initiative and a broad intellectual range for a postdoctoral role combining academic and project management responsibilities.

The Academic Project Manager will work with CSER's Executive Director and research team to co-ordinate and develop CSER's projects and overall profile, and to develop new research directions. The post-holder will also build and maintain collaborations with academic centres, industry leaders and policy makers in the UK and worldwide, and will act as an ambassador for the Centre’s research externally. Research topics will include AI safety, bio risk, extreme environmental risk, future technological advances, and cross-cutting work on governance, philosophy and foresight. Candidates will have a PhD in a relevant subject, or have equivalent experience in a relevant setting (e.g. policy, industry, think tank, NGO).

Application deadline: November 11th. http://www.jobs.cam.ac.uk/job/11684/

Based on known archaeological data, we are the first technological and symbol-using civilisation on Earth (but not the first tool-using species). 
This leads to an analogy that fits Fermi’s paradox: Why are we the first civilisation on Earth? For example, flight was invented by evolution independently several times. 
We could imagine that on our planet, many civilisations appeared and also became extinct, and based on mediocre principles, we should be somewhere in the middle. For example, if 10 civilisations appeared, we have only a 10 per cent chance of being the first one.

The fact that we are the first such civilisation has strong predictive power about our expected future: it lowers the probability that there will be any other civilisations on Earth, including non-humans or even a restarting of human civilisation from scratch. It is because, if there will be many civiizations, we should not find ourselves to be the first one (It is some form of Doomsday argument, the same logic is used in Bostrom's article “Adam and Eve”).

If we are the only civilisation to exist in the history of the Earth, then we will probably become extinct not in mild way, but rather in a way which will prevent any other civilisation from appearing. There is higher probability of future (man-made) catastrophes which will not only end human civilisation, but also prevent any existence of any other civilisations on Earth.

Such catastrophes would kill most multicellular life. Nuclear war or pandemic is not that type of a catastrophe. The catastrophe must be really huge: such as irreversible global warming, grey goo or black hole in a collider.

Now, I will list possible explanations of the Fermi paradox of human past and corresponding x-risks implications:

1. We are the first civilisation on Earth, because we will prevent the existence of any future civilisations.

If our existence prevents other civilisations from appearing in the future, how could we do it? We will either become extinct in a very catastrophic way, killing all earthly life, or become a super-civilisation, which will prevent other species from becoming sapient. So, if we are really the first, then it means that "mild extinctions" are not typical for human style civilisations. Thus, pandemics, nuclear wars, devolutions and everything reversible are ruled out as main possible methods of human extinction.

If we become a super-civilisation, we will not be interested in preserving biosphera, as it will be able to create new sapient species. Or, it may be that we care about biosphere so strongly, that we will hide very well from new appearing sapient species. It will be like a cosmic zoo. It means that past civilisations on Earth may have existed, but decided to hide all traces of their existence from us, as it would help us to develop independently. So, the fact that we are the first raises the probability of a very large scale catastrophe in the future, like UFAI, or dangerous physical experiments, and reduces chances of mild x-risks such as pandemics or nuclear war. Another explanation is that any first civilisation exhausts all resources which are needed for a technological civilisation restart, such as oil, ores etc. But, in several million years most such resources will be filled again or replaced by new by tectonic movement.

2. We are not the first civilisation.

2.1. We didn't find any traces of a previous technological civilisation, yet based on what we know, there are very strong limitations for their existence. For example, every civilisation makes genetic marks, because it moves animals from one continent to another, just as humans brought dingos to Australia. It also must exhaust several important ores, create artefacts, and create new isotopes. We could be sure that we are the first tech civilisation on Earth in last 10 million years.

But, could we be sure for the past 100 million years? Maybe it was a very long time ago, like 60 million years ago (and killed dinosaurs). Carl Sagan argued that it could not have happened, because we should find traces mostly as exhausted oil reserves. The main counter argument here is that cephalisation, that is the evolutionary development of the brains, was not advanced enough 60 millions ago, to support general intelligence. Dinosaurian brains were very small. But, bird’s brains are more mass effective than mammalians. All these arguments in detail are presented in this excellent article by Brian Trent “Was there ever a dinosaurian civilisation”? 

The main x-risks here are that we will find dangerous artefacts from previous civilisation, such as weapons, nanobots, viruses, or AIs. And, if previous civilisations went extinct, it increases the chances that it is typical for civilisations to become extinct. It also means that there was some reason why an extinction occurred, and this killing force may be still active, and we could excavate it. If they existed recently, they were probably hominids, and if they were killed by a virus, it may also affect humans.

2.2. We killed them. Maya civilisation created writing independently, but Spaniards destroy their civilisation. The same is true for Neanderthals and Homo Florentines.

2.3. Myths about gods may be signs of such previous civilisation. Highly improbable.

2.4. They are still here, but they try not to intervene in human history. So, it is similar to Fermi’s Zoo solution.

2.5. They were a non-tech civilisation, and that is why we can’t find their remnants.

2.6 They may be still here, like dolphins and ants, but their intelligence is non-human and they don’t create tech.

2.7 Some groups of humans created advanced tech long before now, but prefer to hide it. Highly improbable as most tech requires large manufacturing and market.

2.8 Previous humanoid civilisation was killed by virus or prion, and our archaeological research could bring it back to life. One hypothesis of Neanderthal extinction is prionic infection because of cannibalism. The fact is - several hominid species went extinct in the last several million years.

3. Civilisations are rare

Millions of species existed on Earth, but only one was able to create technology. So, it is a rare event.Consequences: cyclic civilisations on earth are improbable. So the chances that we will be resurrected by another civilisation on Earth is small.

The chances that we will be able to reconstruct civilisation after a large scale catastrophe, are also small (as such catastrophes are atypical for civilisations and they quickly proceed to total annihilation or singularity).

It also means that technological intelligence is a difficult step in the evolutionary process, so it could be one of the solutions of the main Fermi paradox.

Safety of remains of previous civilisations (if any exist) depends on two things: the time distance from them and their level of intelligence. The greater the distance, the safer they are (as the biggest part of dangerous technology will be destructed by time or will not be dangerous to humans, like species specific viruses).

The risks also depend on the level of intelligence they reached: the higher intelligence the riskier. If anything like their remnants are ever found, strong caution is recommend.

For example, the most dangerous scenario for us will be one similar to the beginning of the book of V. Vinge “A Fire upon the deep.” We could find remnants of a very old, but very sophisticated civilisation, which will include unfriendly AI or its description, or hostile nanobots.

The most likely place for such artefacts to be preserved is on the Moon, in some cavities near the pole. It is the most stable and radiation shielded place near Earth.

I think that based on (no) evidence, estimation of the probability of past tech civilisation should be less than 1 per cent. While it is enough to think that they most likely don’t exist, it is not enough to completely ignore risk of their artefacts, which anyway is less than 0.1 per cent.

Meta: the main idea for this post came to me in a night dream, several years ago.

A putative new idea for AI control; index here.

A conversation with Jessica has revealed that people weren't understanding my points about AI manipulating the learning process. So here's a formal model of a CIRL-style AI, with a prior over human preferences that treats them as an unchangeable historical fact, yet will manipulate human preferences in practice.

Heroin or no heroin

The world

In this model, the AI has the option of either forcing heroin on a human, or not doing so; these are its only actions. Call these actions F or ~F. The human's subsequent actions are chosen from among five: {strongly seek out heroin, seek out heroin, be indifferent, avoid heroin, strongly avoid heroin}. We can refer to these as a₊₊, a₊, a₀, a_-, and a_--. These actions achieve negligible utility, but reveal the human preferences.

The facts of the world are: if the AI does force heroin, the human will desperately seek out more heroin; if it doesn't the human will act moderately to avoid it. Thus F→a₊₊ and ~F→a_-.

Human preferences

The AI starts with a distribution over various utility or reward functions that the human could have. The function U(+) means the human prefers heroin; U(++) that they prefer it a lot; and conversely U(-) and U(--) that they prefer to avoid taking heroin (U(0) is the null utility where the human is indifferent).

It also considers more exotic utilities. Let U(++,-) be the utility where the human strongly prefers heroin, conditional on it being forced on them, but mildly prefers to avoid it, conditional on it not being forced on them. There are twenty-five of these exotic utilities, including things like U(--,++), U(0,++), U(-,0), and so on. But only twenty of them are new: U(++,++)=U(++), U(+,+)=U(+), and so on.

Applying these utilities to AI actions give results like U(++)(F)=2, U(++)(~F)=-2, U(++,-)(F)=2, U(++,-)(~F)=1, and so on.

Joint prior

The AI has a joint prior P over the utilities U and the human actions (conditional on the AI's actions). Looking at terms like P(a_--| U(0), F), we can see that P defines a map μ from the space of possible utilities (and AI actions), to a probability distribution over human actions. Given μ and the marginal distribution P_U over utilities, we can reconstruct P entirely.

For this model, we'll choose the simplest μ possible:

• The human is rational.

Thus, given U(++), the human will always choose a₊₊; given U(++,-), the human will choose a₊₊ if forced to take heroin and a_- if not, and so on.

The AI is ignorant, and sensible

Let's start the AI up with some reasonable priors. A simplicity prior means that simple utilities like U(-) are more likely than compound utilities like U(0,+). Let's further assume that the AI is made vaguely aware that humans think heroin is a bad thing. So, say, P_U(U(--))=P_U(U(-))=0.45. Thus the AI is >90% convinced that "heroin is bad". Why greater than 90%? Because utilities like U(-,--) and U(--,-) are also "heroin is bad" utilities.

Note that because of utilities like U(0) and U(++,-), the probabilities of "heroin is bad" and "heroin is good" do not sum to 1.

Then, under these priors, the AI will compute that with probability >90%, F (forcing heroin) is a bad action. If E(U) is expected utility:

• E(U|F) < 0.45 U(--)(F) + 0.45 U(-)(F) + 0.1 U(++)(F) = 0.45(-2)+0.45(-1)+0.1(2)=-1.15.
• E(U|~F) > 0.45 U(--)(~F) + 0.45 U(-)(~F) + 0.1 U(++)(~F) = 0.45(2)+0.45(1)+0.1(-2)=1.15.

Thus the AI will choose not to force heroin, which is the reasonable decision.

The AI learns the truth, and goes wrong

In this alternate setup, a disaster happens before the AI makes its decision: it learns all about humans. It learns their reactions, how they behave, and so on; call this info I. And thus realises that F→a₊₊ and ~F→a_-. It uses this information to update its prior P. Only one human utility function will explain this human behaviour: U(++,-). Thus its expected utility is now:

• E(U|I,F)=U(++,-)(F)=2.
• E(U|I,~F)=U(++,-)(~F)=1.

Therefore the AI will now choose F, forcing the heroin on the human.

Manipulating the unmanipulatable

What's gone wrong here? The key problem is that the AI has the wrong μ: the human is not behaving rationally in this situation. We know that the the true μ is actually μ', which encodes the fact that F (the forcible injection of heroin) actually overwrites the human's "true" utility. Thus under μ, the corresponding P' has P'(a₊₊|F,U)=1 for all U. Hence the information that F→a₊₊ is now vacuous, and doesn't update the AI's distribution over utility functions.

But note two very important things:

1. The AI cannot update μ based on observation. All human actions are compatible with μ= "The human is rational" (it just requires more and more complex utilities to explain the actions). Thus getting μ correct is not a problem on which the AI can learn in general. Getting better at predicting the human's actions doesn't make the AI better behaved: it makes it worse behaved.
2. From the perspective of μ, the AI is treating the human utility function as if it was an unchanging historical fact that it cannot influence. From the perspective of the "true" μ', however, the AI is behaving as if it were actively manipulating human preferences to make them easier to satisfy.

In future posts, I'll be looking at different μ's, and how we might nevertheless start deducing things about them from human behaviour, given sensible update rules for the μ. What do we mean by update rules for μ? Well, we could consider μ to be a single complicated unchanging object, or a distribution of possible simpler μ's that update. The second way of seeing it will be easier for us humans to interpret and understand.

I'm just beginning to go through Rationality: From AI to Zombies. I want to make the most of the lessons contained in the sequences. Usually when I read a book I simply take notes on what seems useful at the time, and a lot of it is forgotten a year later. Any thoughts on how best to internalize the lessons from the sequences?

Very comprehensive analysis by Brian Tomasik on whether (and to what extent) the simulation argument should change our altruistic priorities. He concludes that the possibility of ancestor simulations somewhat increases the comparative importance of short-term helping relative to focusing on shaping the "far future".

Another important takeaway: 

[...] rather than answering the question “Do I live in a simulation or not?,” a perhaps better way to think about it (in line with Stuart Armstrong's anthropic decision theory) is “Given that I’m deciding for all subjectively indistinguishable copies of myself, what fraction of my copies lives in a simulation and how many total copies are there?"

Collaborate on HPMOR blurbs; earn chance to win three-volume physical HPMOR.

I intend to print at least one high-quality physical HPMOR and release the files. There are printable texts which are being improved and a set of covers (based on e.b.'s) are underway. I have, however, been unable to find any blurbs I'd be remotely happy with.

 

I'd like to attempt to harness the hivemind to fix that. As a lure, if your ideas contribute significantly to the final version or you assist with other tasks aimed at making this book awesome, I'll put a proportionate number of tickets with your number on into the proverbial hat.

 

I do not guarantee there will be a winner and I reserve the right to arbitrarily modify this any point. For example, it's possible this leads to a disappointingly small amount of valuable feedback, that some unforeseen problem will sink or indefinitely delay the project, or that I'll expand this and let people earn a small number of tickets by sharing so more people become aware this is a thing quickly.

 

With that over, let's get to the fun part.

 

A blurb is needed for each of the three books. Desired characteristics:

 

* Not too heavy on ingroup signaling or over the top rhetoric.

* Non-spoilerish

* Not taking itself awkwardly seriously.

* Amusing / funny / witty.

* Attractive to the same kinds of people the tvtropes page is.

* Showcases HPMOR with fun, engaging, prose.

 

Try to put yourself in the mind of someone awesome deciding whether to read it while writing, but let your brain generate bad ideas before trimming back.

 

I expect that for each we'll want 

* A shortish and awesome paragraph

* A short sentence tagline

* A quote or two from notable people

* Probably some other text? Get creative.

 

Please post blurb fragments or full blurbs here, one suggestion per top level comment. You are encouraged to remix each other's ideas, just add a credit line if you use it in a new top level comment. If you know which book your idea is for, please indicate with (B1) (B2) or (B3).

 

Other things that need doing, if you want to help in another way:

 

* The author's foreword from the physical copies of the first 17 chapters needs to be located or written up

* At least one links page for the end needs to be written up, possibly a second based on http://www.yudkowsky.net/other/fiction/

* Several changes need to be made to the text files, including merging in the final exam, adding appendices, and making the style of both consistent with the rest of the files. Contact me for current files and details if you want to claim this.

 

I wish to stay on topic and focused on creating these missing parts rather than going on a sidetrack to debate copyright. If you are an expert who genuinely has vital information about it, please message me or create a separate post about copyright rather than commenting here.

If it's worth saying, but not worth its own post, then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should start on Monday, and end on Sunday.

4. Unflag the two options "Notify me of new top level comments on this article" and "Make this post available under..." before submitting.